I'd like to extract a stream of data from a live webpage updated continuously by a javascript code that doesn't reload the page itself. How do I do this with selenium if it's possible?
1 Answer
You can find a new/added element in a loop. It could look like this in Java:
WebElement element = driver.findElement(By.id("theId"));
int count = 0;
while (element != null && count < 60) {
// get the element again
element = driver.findElement(By.id("theId"));
// do something with the element...
log(element.text());
// wait one second
Thread.sleep(1000);
// increment a count if you want to eventually escape this loop, in case you never find your elment
count++;
}
If that doesn't work for your purpose, you can get the full source code of the page and parse it for whatever information you are looking for.
String pageSource = driver.getPageSource();
You can do that in a loop like above, and keep trying until you reach a timeout or find the updated info.
1 Comment
Dingredient
If this answer doesn't help, can you provide more detail? Maybe you can show the JavaScript code which does the updating, or the part of the page code that gets updated?