Flutter: Fetch certain elements of a HTML page

Question

I recently started developing an app for an animal shelter using dart & flutter, and ran into problem.

The idea is that there is a donation page, where the user can choose to buy food for the dogs. The opening scaffold would show the picture of the food, and fetch some data from a website that sells said food, and the current price. The image is stored locally as an asset, but I'd like to fetch at least the price from the website, so it is always up to date.

I followed the tutorial on flutter.io, but the problem is that the http.get returns the whole code of the site, not just the portion I specified (description_tab). I tried checking the async & dart:convert documentation, but so far I haven't found anything that could help in scraping specific content from a HTML site, using the div tag.

The full code can be found here: https://github.com/kergefarkas/osszefogasaszanhuzokert

This is how I was able to fetch the whole HTML code of the site:

Future<JoseraAdultActive> fetchPost() async {
  final response = await http.get(
      'https://www.petissimo.hu/kutyaknak/szarazeledelek/josera/josera-active.html');
  final json = JSON.decode(response.body);

  return new JoseraAdultActive.fromJson(json);
}

class JoseraAdultActive {
  final String description;
  final String price;

  JoseraAdultActive({this.description, this.price});

  factory JoseraAdultActive.fromJson(Map<String, dynamic> json) {
    return new JoseraAdultActive(
        description: json['description_tab'],
        price: json['product_price_from']);
  }
}

And this is where I would display the extracted information:

new Container(
                    padding: const EdgeInsets.only(top: 10.0, right: 5.0),
                    child: new FutureBuilder<JoseraAdultActive>(
                        future: fetchPost(),
                        builder: (context, snapshot) {
                          if (snapshot.hasData) {

                            return new Text('');
                          } else if (snapshot.hasError) {
                            return new Text('Error');
                          }
                          return new CircularProgressIndicator();
                        })),

I also think that the JSON.decode won't work with a HTML code, but couldn't find anything to use instead. Trying to slice it via substrings didn't work either.

Thanks for any help in advance!

Would github.com/dart-lang/html work?

Seth Ladd
– Seth Ladd

2018-03-21 16:24:27 +00:00
Commented Mar 21, 2018 at 16:24 — Seth Ladd
– Seth Ladd, Commented Mar 21, 2018 at 16:24

rmtmckenzie · Accepted Answer · 2018-03-21 22:50:33Z

8

Furthering Seth's comment, you should be able to use the html library for this.

That should look something like this:

import 'package:html/parser.dart' show parse;
import 'package:html/dom.dart';

var document = parse(response.body);
var priceElement = document.getElementsByClassName("product_price_from");
// priceElement may have weird formatting (I see &nbsp;) so you might need to do some parsing
// here to solve that

I'd also recommend learning about Regular Expressions (RegEx) as they could be used to solve this fairly simply as well and is useful for all sorts of programming problems. Here is a link to dart's regex documentation and a general RegEx tutorial.

That being said, this code will be dependent on the URL changing, the website changing or being down, that particular product not being sold anymore, and will use the user's data, and many other factors that may or may not be under your control - so I'd be a bit leery about putting it in production code even if it's just for an app for a shelter. At least make sure it has a safe fallback (i.e. fallback to the current price of 11500 Ft).

I'd give a good thought as to whether this is actually necessary, rather than simply updating it yourself once every year if/when the price changes. Or if you have a server backend for your app (or even access to the shelter's website - you could put a page at shelter.com/donationprices/joseraadultactive with just "11499" being returned), I'd recommend making this just a parameter the app can retrieve that you can update (and if you absolutely want it to be done automatically, have a task that runs on the server once a day to retrieve the newest price). Also, I don't know how shelters work where you are, but where I live they tend to buy food in bulk at larger quantities or when it's on sale so the price for one bag of food is somewhat irrelevant - at that point you may as well just have a donate button with a picture of a bag and an arbitrary price as you'll probably get just as many donations, and many more than if there is a problem with the app.

answered Mar 21, 2018 at 22:50

rmtmckenzie

40.7k10 gold badges121 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Zoltán Györkei Over a year ago

Thank you, I'll try this. I also thought about the fail safe, and was planning to add it. Currently I don't have any webserver, and want to build this app and enhance things like fetching the price separately from my own source later. I was never really good at developing, and after the uni I went to testing, where you don't face many coding related challenges. So this is my learning curve, and I'm happy to take any advice :)

rmtmckenzie Over a year ago

My best advice is a common moniker especially in software - KISS (keep it simple, stupid) i.e. if there's a simpler way of doing it, do it that way =D. And with apps where 99% of users auto-update, it's so easy to push new features out that it's often a good idea to start with the simplest way of doing it (i.e. hard coding the value) and finishing the rest of the app, getting it released, and then working on adding additional functionality. This would also be a candidate for Firebase remote config but no flutter plugin for that yet =(. But if this is mostly for learning then go for it!

Collectives™ on Stack Overflow

Flutter: Fetch certain elements of a HTML page

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related