Get JSON object one by one from JSON array in Android using Retrofit and RxJava

Question

I am using retrofit to hit my network api which returns json array. I am doing so using following code-

Observable<MyJson[]> response = (Observable<MyJson[]>)mNetworkService.getReadyObserverable(mNetworkService.getNetworkServiceApi().getMyDataJsons(), MyJson.class, true, useCache);
        mAirlineSubscription = response.subscribe(new Observer<MyJson[]>() {
            @Override
            public void onCompleted() {
                Log.d(TAG, "getData completed..");
            }

            @Override
            public void onError(Throwable e) {
                Log.e(TAG, "onError: "  + e.getLocalizedMessage());
            }

            @Override
            public void onNext(MyJson[] myJsonData) {
                //I want here json data one by one
            }

But my problem is json data array get downloaded completely and then only onNext gets called. Here I want onNext should get called when retrofit downloads first json object from myData[] json array and keep continue until my all json objects of myData[] get downloaded. This way my UI would look more responsive and better in terms of user interaction.

Can anyone help to fix this?

Retrofit does not support this out of the box - custom CallAdapter and Converter are needed. Both are required - in CallAdapter, ResponseBody's stream must be processed (get individual array items) AND converted to java objects, and Converter should just pass those results as is (noop). First, this contradicts retrofit's design, where CallAdapter and Converter are independent entities.Second, implementation would be nontrivial and lengthy - json streaming parsing, rxjava backpressure etc - i suggest you to find another solution to problem - data paging, smaller response etc — m.ostroverkhov
– m.ostroverkhov, Commented Sep 5, 2016 at 14:28

Alexander Skvortsov · Accepted Answer · 2016-11-14 21:04:50Z

3

Receiving elements One by One, as fast as possible?
Don't want to wait when everything will be downloaded?

The solution is: Declare the method of retrofit's interface with annotation @Streaming and use Observable<ResponseBody> as return value. And then, by using flatMap(), convert ResponseBody to series of POJO (Observable<TYPE>).

Example:

Declare retrofit interface:

public interface HugeJsonApi {

String SERVICE_ENDPOINT = "https://raw.githubusercontent.com";

@Streaming
@GET("/zemirco/sf-city-lots-json/master/citylots.json")
Observable<ResponseBody> get();
}

Use it like that:

public void playHugeJsonSample() {

    HugeJsonApi hugeJsonApi = RestUtils.createService(HugeJsonApi.class, HugeJsonApi.SERVICE_ENDPOINT);

    Handler handler = new Handler(Looper.getMainLooper());
    hugeJsonApi.get()
            .flatMap(responseBody -> convertObjectsStream(responseBody, gson, Feature.class))
            .subscribeOn(Schedulers.io())
            .observeOn(AndroidSchedulers.mainThread())
            .subscribe(new Subscriber<Feature>() {

                @Override
                public void onStart() {
                    super.onStart();
                    request(1);
                }

                @Override
                public void onNext(Feature feature) {
                    Log.i(TAG, gson.toJson(feature));
                    counter[0]++;
                    request(1);
                }

                @Override
                public void onCompleted() {
                    Log.i(TAG, "onCompleted() called. Fetched elements:" + counter[0]);
                }

                @Override
                public void onError(Throwable e) {
                    Log.e(TAG, "something went wrong", e);
                }
            });
}

@NonNull
private static <TYPE> Observable<TYPE> convertObjectsStream(ResponseBody responseBody, Gson gson, Class<TYPE> clazz) {
    Type type = TypeToken.get(clazz).getType();
    return Observable.create(SyncOnSubscribe.<JsonReader, TYPE>createStateful(
            // initialize the reader
            () -> {
                try {
                    JsonReader reader = gson.newJsonReader(responseBody.charStream());
                    reader.beginObject();
                    return reader;
                } catch (IOException e) {
                    e.printStackTrace();
                    RxJavaHooks.onError(e);
                }
                return null;
            },
            // read elements one by one
            (reader, observer) -> {

                if (reader == null) {
                    observer.onCompleted();
                    return null;
                }

                try {
                    if (reader.hasNext()) {
                        TYPE t = gson.fromJson(reader, type);
                        observer.onNext(t);
                    }
                    else {
                        observer.onCompleted();
                    }

                } catch (IOException e) {
                    e.printStackTrace();
                    observer.onError(e);
                }

                return reader;
            },
            // close the reader
            reader -> {
                if (reader != null) {
                    try {
                        reader.close();
                    } catch (Exception e) {
                        e.printStackTrace();
                        RxJavaHooks.onError(e);
                    }
                }
            }

    ));
}

Here is workable example:

https://github.com/allco/RetrofitAndRxJava

It takes 180+Mb Json and parses it as a real stream.

edited Nov 14, 2016 at 21:04

answered Oct 22, 2016 at 0:26

Alexander Skvortsov

2,7742 gold badges21 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Alexander Skvortsov Over a year ago

@AndroDev, do you have any feedback?

AndroDev Over a year ago

not yet tried. was on vacation. will surely update you once I check.

Alexander Skvortsov Over a year ago

Changed a bit the implementation according stackoverflow.com/documentation/rx-java/2341/backpressure/10629/…

rawcoder064 Over a year ago

What if i have another json array in same response like "features" exp: {"features":[], "images":[]} How do i parse this as a stream?

Alexander Skvortsov Over a year ago

Well, all the magic happens at convertObjectsStream. Take a look at the middle lambda (reader, observer) -> {...}, open there an array with JsonReader.beginArray() and open then another one when the first will be exhausted. But then "features" and "imges" should be represented by the same Java interface/class (in this example by Feature.class) in order to be able to pass data further with observer.onNext(t). Probably GsonDecoder could help here as well - the one which knows how to translate "features" and "images" to Feature.class

|

Matias Elorriaga · Accepted Answer · 2016-09-05 13:18:32Z

1

your service should look like this:

@GET("users/{username}/repos")
Observable<List<Repository>> publicRepositories(@Path("username") String username);

I mean, return a Observable<List<Something>>

answered Sep 5, 2016 at 13:18

Matias Elorriaga

9,1585 gold badges44 silver badges60 bronze badges

1 Comment

m.ostroverkhov Over a year ago

OP wants list items as soon as they available - Observable<Repository> ,this approach still downloads all response and then issues single notification - Observable<List<Repository>>

Marjan Davodi · Accepted Answer · 2017-10-24 10:03:32Z

i have same question i explain it base of own project, you should use my retrofit request is

@GET("StudentApi/getAll")
    Observable<List<Person>> getPersons();

and i have an interface to run server request and it has this method for run getPersons() and return list of json array of persons info

Observable<Person> getAllPersons();

and the important part is body of up method and should like bellow

@Override
    public Observable<Person> getAllPersons() {
        Observable<List<Person>> observable =
                serviceGenerator.getService().getPersons();
        return observable
                .flatMap(new Function<List<Person>, Observable<Person>>() {
                    @Override
                    public Observable<Person> apply(List<Person> persons) throws Exception {
                        return Observable.fromIterable(persons);
                    }
                });
}

i found that fromIterable method return objects of json array one by one and finally in an activity i get json objects from up methods like bellow

public void getPersons(){
      personsRepository.getAllPersons()
                .observeOn(AndroidSchedulers.mainThread())
                .subscribeOn(Schedulers.io())
                .subscribeWith(new DisposableObserver<Person>() {
                    @Override
                    public void onNext(Person value) {
                        Log.e("person info  is",value.getpName() + value.getpFamily());
                    }

                    @Override
                    public void onError(Throwable throwable) {
                        Log.e("onError",throwable.toString());
                    }

                    @Override
                    public void onComplete() {
                        Log.e("onComplete","onComplete");
                    }
                });
    }

i hope is useful :)

Collectives™ on Stack Overflow

Get JSON object one by one from JSON array in Android using Retrofit and RxJava

3 Answers 3

6 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related