When JSON has URLs but you need data (or how to chain API calls and merging the result)

The Problem

Let's say you have an API (ie. http://api.example.org/stuffs) that returns an array of stuff. But the API dev who worked on it, the bastard that he is, decided that certain fields, like say price is a hyperlink. So it looks like:

[{
   "id": "exde01"
   "name": "Stuff1",
   "type": "just stuff",
   "weight": "3.22",
   "price": "http://api.example.org/price/exde01"
},
{
   "id": "exde02"
   "name": "Stuff2",
   "type": "just stuff",
   "weight": "3.25",
   "price": "http://api.example.org/price/exde02"
}
.....
]

Ok. Now when you do an API call to the price url you'd get something like:

{
   "inlu_price": 69.96
   "exlu_tax": 49.99,
   "price": 123.25
}

So now the problem is how do we loop over the first JSON result with stuff and calling each price URL and then appending back the result. Essentially, we want the end result to look like this:

[{
   "id": "exde01"
   "name": "Stuff1",
   "type": "just stuff",
   "weight": "3.22",
   "price": {
     "inlu_price": 69.96
     "exlu_tax": 49.99,
     "price": 123.25
   }
},
{
   "id": "exde02"
   "name": "Stuff2",
   "type": "just stuff",
   "weight": "3.25",
   "price": {
     "inlu_price": 69.96
     "exlu_tax": 49.99,
     "price": 123.25
   }
}
.....
]

The Solution

My solution is in TypeScript and if it wasn't for reactive extensions (or streams) I'd probably have hanged myself. 

The first step is to get the main JSON. So I wrote a service for it.

    getAllStuff(): Observable {
        const url = "http://api.example.org/stuffs";

        return this.http.get(url}).map((response) => response.json());
    }

Now for the tricky part.

   this.restService.getAllStuff().switchMap((stuffs) => {
      return Observable.forJoin(stuffs.map((s) => {
         return this.http.get(s.price).map((res) => {
            s.price = res;
            return s;
         }
      }
   })
   .subscribe((r) => {
      console.log(JSON.stringify(r));
   });

This looks hard but it isn't really once you understand how it "flows".

First the getAllStuff() call will return the first set of results (technically, an Observable array) which we pass in into the forJoin() function.

What forJoin() does it take an array of API calls (as Observables) and spits out a single "merged" result. That's where the stuffs.map() does. But take note this is a two step process.

If you just focus on just the stuffs.map() . You can access stuff JSON array and drill down to the s.price, ie. ['http://api.example.org/price/exde01', 'http://api.example.org/price/exde02'].

This is why then we do a return with the this.http.get() function to produce that array forJoin() needs. ForJoin() then resolves these calls. After resolving, the result is passed back up to the switchMap() function which we can then subscribe to get the transformed JSON.

There we get our desired JSON result albeit with a bit of processing.

Now, if there's a 2nd URL in your main json, then you'll just need another switchMap(). You can chain multiple switchMaps().

References:

• http://blog.danieleghidoli.it/2016/10/22/http-rxjs-observables-angular/
• https://www.learnrxjs.io/operators/transformation/switchmap.html
• https://stackoverflow.com/questions/38517552/angular-2-chained-http-get-requests-with-iterable-array