During the Riley Children’s Health web development project, our team implemented a fresh design, layout and CMS (content management system). Nothing was left off the table during our overhaul—except for one thing: displaying content from Riley at IU Health’s internal database onto the website. This database contained proprietary information that needed to be shown on the website, but changing the structure or the business logic of how we could pull information was not feasible for this launch. So, we had to find a way to ensure that this information would be displayed correctly on the front-end of the website for users without compromising the sources of the data on the back-end. The Challenge During the content curation phase, there were a few databases from different data sources in addition to our client’s internal database. Some hypothetical examples of a data source could be SharePoint, a CMS, Apache Solr or Google Custom Search Engine (Google CSE). These new data sources contained APIs intended to be used not only on rileychildrens.org but also on other IU Health websites. Also, some of the data from the different data sources were closely related and needed to be displayed together seamlessly on the front-end. These data sources were temporary, so we also needed the ability to swap out a data source without having to rewrite the front-end code to cater to the new data source’s proprietary schema. Our Approach We decided to build a custom application that would connect to all of the data sources via their own provided APIs. Each data source was categorized with a type. For example, the Apache Solr and Google CSE data sources would be categorized as a “search engine” inside the application. Each type had a standardized set of URL parameters that its API endpoints would accept as well as a standardized JSON output. We took the output from the data sources, and we modified it in a way that developers could switch out a data source while keeping the final API endpoints’ result structure exactly the same, as illustrated below.To illustrate how this works, suppose we wanted to swap out a search engine (i.e., data source) for a different one. A standardized URL and its output would look like this:Because of this, the final choice for the search engine wouldn’t matter. Why? We could manipulate the data source using the custom API to output the desired results regardless of the input data. For example, Google CSE will give these results from the following endpoint:Whether we decided to use Google CSE or another search engine, the front-end code would make an AJAX call to: (https://rileyapi.com/query=heart&per_page=3&page=1). Our API takes the parameters in the endpoint’s URL and converts them to the search engine’s parameters. Then we concatenate the search engine’s URL and the parameters to form the correct URL. For example, the parameters’ conversion for Google CSE to our API looks something like this:After we convert the parameters to match what Google CSE is looking for, we use the completed URL to get the desired results. The Benefits of a Custom API There are a couple of clear benefits of using this custom API structure. First, the front-end won’t affect the back-end because we kept separation of concerns (SoC) in mind when we designed this workflow. Even if a data source is switched out, the results structure will be the same. The front-end only cares about the final result structure. Through simple configuration, we can change the data source in just a couple lines of code, which is useful if a client would like to replace one system with another at any point in time. Organizing our code this way offers flexibility in choosing data sources. We don’t need to worry if data sources are temporary or permanent, nor do we need to constantly update the front-end code to accommodate every data source change. The second benefit is that we were able to combine the output of data source API endpoints. In our case, we had two sets of related data from two different sources that needed to be displayed together. We were able to call both of the API endpoints and combine the output to represent the data fully and cleanly on the front-end. The third benefit is that we now have a custom application that channels data all in one place. Other IU Health web properties, then, can make use of this API without the need for extra plugins. It is CMS independent. Notes on Possible Solutions There is a lot of power and flexibility in creating a custom API. Often, there is data that needs to be displayed on a website, but what houses that data could change. Creating a custom API allowed us to develop our front-end solution simultaneously while our back-end data sources were in flux. When we needed to switch out a data source for another, we didn’t have to worry about the effects it would have on the front-end. It just worked! While the optimal solution is to modify the original data source, that is not often possible. The second solution, then, would be to create a new data source from scratch to have total control over its design and implementation. But, if neither of those solutions are feasible, creating a custom API that modifies existing data sources—which was our solution for the Riley at IU Health project—is another choice that still offers customizability and control over the data. To find out more about the web development process for the Riley at IU Health website project, learn why we chose Craft as the CMS (content management system) or read more about how custom search can improve customer experience.

Development

Building a Custom API to Deliver Relevant Search Results


October 18, 2016

During the Riley Children’s Health web development project, our team implemented a fresh design, layout and CMS (content management system). Nothing was left off the table during our overhaul—except for one thing: displaying content from Riley at IU Health’s internal database onto the website. This database contained proprietary information that needed to be shown on the website, but changing the structure or the business logic of how we could pull information was not feasible for this launch.

So, we had to find a way to ensure that this information would be displayed correctly on the front-end of the website for users without compromising the sources of the data on the back-end.

The Challenge

During the content curation phase, there were a few databases from different data sources in addition to our client’s internal database. Some hypothetical examples of a data source could be SharePoint, a CMS, Apache Solr or Google Custom Search Engine (Google CSE). These new data sources contained APIs intended to be used not only on rileychildrens.org but also on other IU Health websites. Also, some of the data from the different data sources were closely related and needed to be displayed together seamlessly on the front-end. These data sources were temporary, so we also needed the ability to swap out a data source without having to rewrite the front-end code to cater to the new data source’s proprietary schema.

Our Approach

We decided to build a custom application that would connect to all of the data sources via their own provided APIs. Each data source was categorized with a type. For example, the Apache Solr and Google CSE data sources would be categorized as a “search engine” inside the application. Each type had a standardized set of URL parameters that its API endpoints would accept as well as a standardized JSON output.

We took the output from the data sources, and we modified it in a way that developers could switch out a data source while keeping the final API endpoints’ result structure exactly the same, as illustrated below.

Blog Riley Api Image1
We combined different data sources in our custom API to output results that displayed consistently on the front-end.

To illustrate how this works, suppose we wanted to swap out a search engine (i.e., data source) for a different one. A standardized URL and its output would look like this:

    
      (https://rileyapi.com/query=heart&per_page=3&page=1)

{  
  "results":{  
     "docs":[  
        {  
           "url":"https://www.rileychildrens.org/departments/riley-heart-center",
           "description":"As the engine of the human body, the heart and cardiovascular system are \ncentral to life. When your child or loved one is born with a heart defect, an \nintegrated ...",
           "title":"Riley Heart Center | Riley at IU Health"
        },
        {  
           "url":"https://www.rileychildrens.org/departments/cardiothoracic-surgery",
           "description":"Children and adults with congenital heart disease receive highly specialized care \nby cardiovascular and cardiovascular surgeons at Riley at IU Health.",
           "title":"Cardiothoracic Surgery | Riley at IU Health"
        },
        {  
           "url":"https://www.rileychildrens.org/departments/adults-with-congenital-heart-disease",
           "description":"An estimated one million adults in the United States are living with congenital \nheart disease, according to the Centers for Disease Control and Prevention.",
           "title":"Adults with Congenital Heart Disease | Riley at IU Health"
        }
     ],
     "meta":{  
        "page":1,
        "per_page":3,
        "total_entries":357,
        "total_pages":120
     }
  }
}
    
  

Because of this, the final choice for the search engine wouldn’t matter. Why? We could manipulate the data source using the custom API to output the desired results regardless of the input data. For example, Google CSE will give these results from the following endpoint:

    
      (https://www.googleapis.com/cus... key]&cx=[app id]&q=heart&num=3&start=1&fields=searchInformation/totalResults,items(title,snippet,formattedUrl,pagemap/cse_thumbnail/src))
 
{
"searchInformation": {
 "totalResults": "382"
},
"items": [
 {
  "title": "Cardiology | Riley at IU Health",
  "snippet": "Cardiologists plan and manage even the most complex care for congenital heart \ndefects and acquired heart disease at Riley at IU Health Cardiology.",
  "formattedUrl": "https://www.rileychildrens.org/departments/cardiology"
},
 {
  "title": "Hypoplastic Left Heart Syndrome | Riley at IU Health",
  "snippet": "The heart performs the critical job of pumping blood to and from the lungs and \nthroughout the body. Because its work never stops, this pump must be strong ...",
  "formattedUrl": "https://www.rileychildrens.org/.../hypoplastic-left-heart-syndrome"
},
 {
  "title": "Congenital Heart Disease | Riley at IU Health",
  "snippet": "Newborns with congenital heart disease require specialized care in a NICU. \nLearn how Riley at IU Health treats this condition.",
  "formattedUrl": "https://www.rileychildrens.org/health-info/congenital-heart-disease"
}
]
}
    
  

Whether we decided to use Google CSE or another search engine, the front-end code would make an AJAX call to: (https://rileyapi.com/query=heart&per_page=3&page=1).

Our API takes the parameters in the endpoint’s URL and converts them to the search engine’s parameters. Then we concatenate the search engine’s URL and the parameters to form the correct URL.

For example, the parameters’ conversion for Google CSE to our API looks something like this:

Blog Riley Api Image2
In this scenario, the user searched for "heart" on the website (as seen within the URLs in the "Example URL" row).

After we convert the parameters to match what Google CSE is looking for, we use the completed URL to get the desired results.

The Benefits of a Custom API

There are a couple of clear benefits of using this custom API structure.

First, the front-end won’t affect the back-end because we kept separation of concerns (SoC) in mind when we designed this workflow. Even if a data source is switched out, the results structure will be the same. The front-end only cares about the final result structure. Through simple configuration, we can change the data source in just a couple lines of code, which is useful if a client would like to replace one system with another at any point in time. Organizing our code this way offers flexibility in choosing data sources. We don’t need to worry if data sources are temporary or permanent, nor do we need to constantly update the front-end code to accommodate every data source change.

The second benefit is that we were able to combine the output of data source API endpoints. In our case, we had two sets of related data from two different sources that needed to be displayed together. We were able to call both of the API endpoints and combine the output to represent the data fully and cleanly on the front-end.

The third benefit is that we now have a custom application that channels data all in one place. Other IU Health web properties, then, can make use of this API without the need for extra plugins. It is CMS independent.

Notes on Possible Solutions

There is a lot of power and flexibility in creating a custom API. Often, there is data that needs to be displayed on a website, but what houses that data could change. Creating a custom API allowed us to develop our front-end solution simultaneously while our back-end data sources were in flux. When we needed to switch out a data source for another, we didn’t have to worry about the effects it would have on the front-end. It just worked!

While the optimal solution is to modify the original data source, that is not often possible. The second solution, then, would be to create a new data source from scratch to have total control over its design and implementation. But, if neither of those solutions are feasible, creating a custom API that modifies existing data sources—which was our solution for the Riley at IU Health project—is another choice that still offers customizability and control over the data.


To find out more about the web development process for the Riley at IU Health website project, learn why we chose Craft as the CMS (content management system) or read more about how custom search can improve customer experience.