Reading HTML content into java program

Question

I am trying to get recharge plan information of service provider into my java program, the website contains dynamic data, and when i am fetching the URL using URLConnection i am only getting the static content,I want to automate the recharge plans of different website into my program.

package com.fs.store.test;
import java.net.*;
import java.io.*;

public class MyURLConnection 
{

    private static final String baseTataUrl = "https://www.tatadocomo/pre-paypacks";`enter code here`

    public MyURLConnection()
    {

    }

    public void getMeData() 
    {
        URLConnection urlConnection = null;
        BufferedReader in = null;
            try
                {
                    URL url = new URL(baseTataUrl);
                    urlConnection = url.openConnection();
                    HttpURLConnection connection = null;
                     connection = (HttpURLConnection) urlConnection;

                         in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()/*,"UTF-8"*/));
                         String currentLine = null;

                    StringBuilder line = new StringBuilder();

                         while((currentLine = in.readLine()) != null)
                         {  
                              System.out.println(currentLine);
                              line = line.append(currentLine.trim());
                         } 
                      }catch(IOException e)
                      {
                         e.printStackTrace();
                      }
                        finally{
                            try{
                                in.close();
                            }
                            catch(Exception e){
                                e.printStackTrace();
                            }
                        }
        } 

    public static void main (String args[])
    {
        MyURLConnection test = new MyURLConnection();
        System.out.println("About to call getMeData()");
        test.getMeData();
    }


}

Can we see the Java code that you have tried to read the HTML please. — blackpanther
– blackpanther, Commented Apr 2, 2014 at 11:33

416E64726577 · Accepted Answer · 2014-05-17 21:19:35Z

1

You must use one of HtmlEditorKits with Javascript enabled in your browser and then get content. See examples: oreilly

edited May 17, 2014 at 21:19

416E64726577

2,2242 gold badges26 silver badges49 bronze badges

answered Apr 2, 2014 at 11:58

user3436761

114 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Joop Eggen · Accepted Answer · 2014-04-02 12:09:14Z

0

Inspect the traffjc. Firefox has a TamperData plugin for instance. Then you may communicate more directly.

Use apache's HttpClient to facilitate the communication, instead of plain URL.

Maybe use some JSON library if JSON data are coming back.

More details, but you might now skip some loading.

answered Apr 2, 2014 at 12:09

Joop Eggen

110k8 gold badges89 silver badges142 bronze badges

Collectives™ on Stack Overflow

Reading HTML content into java program

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related