Web automation using .NET

Check out SimpleBrowser, which is a fairly mature, lightweight browser automation library.

https://github.com/axefrog/SimpleBrowser

From the page:

SimpleBrowser is a lightweight, yet highly capable browser automation engine designed for automation and testing scenarios. It provides an intuitive API that makes it simple to quickly extract specific elements of a page using a variety of matching techniques, and then interact with those elements with methods such as Click(), SubmitForm() and many more. SimpleBrowser does not support JavaScript, but allows for manual manipulation of the user agent, referrer, request headers, form values and other values before submission or navigation.

If you want to simulate a real browser then WatiN will be a good fit for you. (Selenium is another alternative, but I do not recommend it for you).

If you want to work on the HTTP level, then use WebRequest and related classes.

You could use Selenium WebDriver.

A quick code sample below:

using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;

// Requires reference to WebDriver.Support.dll
using OpenQA.Selenium.Support.UI;

class GoogleSuggest
{
    static void Main(string[] args)
    {
        // Create a new instance of the Firefox driver.
        // Note that it is wrapped in a using clause so that the browser is closed 
        // and the webdriver is disposed (even in the face of exceptions).

        // Also note that the remainder of the code relies on the interface, 
        // not the implementation.

        // Further note that other drivers (InternetExplorerDriver,
        // ChromeDriver, etc.) will require further configuration 
        // before this example will work. See the wiki pages for the
        // individual drivers at http://code.google.com/p/selenium/wiki
        // for further information.
        using (IWebDriver driver = new FirefoxDriver())
        {
            //Notice navigation is slightly different than the Java version
            //This is because 'get' is a keyword in C#
            driver.Navigate().GoToUrl("http://www.google.com/");

            // Find the text input element by its name
            IWebElement query = driver.FindElement(By.Name("q"));

            // Enter something to search for
            query.SendKeys("Cheese");

            // Now submit the form. WebDriver will find the form for us from the element
            query.Submit();

            // Google's search is rendered dynamically with JavaScript.
            // Wait for the page to load, timeout after 10 seconds
            var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
            wait.Until(d => d.Title.StartsWith("cheese", StringComparison.OrdinalIgnoreCase));

            // Should see: "Cheese - Google Search" (for an English locale)
            Console.WriteLine("Page title is: " + driver.Title);
        }
    }
}

The great thing (among others) about this approach is that you can easily switch the underlying browser implementations, just by specifying a different IWebDriver, like FirefoxDriver, InternetExplorerDriver, ChromeDriver, etc. This also means you can write 1 test and run it on multiple IWebDriver implementations, thus testing how the page works when viewed in Firefox, Chrome, IE, etc. People working in QA sector often use Selenium to write automated web page tests.

I'm using ObjectForScripting to automate WebBrowser, A Javascript callback to C# function and then function in c# extract data or automate many-thing.

I have clearly explained in the following link

Web Automation using Web Browser and C#

.NET does not have any built-in functionality for this. It does have the WebClient and HttpRequest/HttpResponse classes, but they are only building blocks.

You cannot easily automate client-side activity, like filling out forms or clicking on buttons from C#. However, if you look into JavaScript, you may be able to better automate some of those things. To really automate, you would need to reverse engineer the call made by clicking the button, and connect to the url directly, using the classes @John mentions.