Search our Blogs
Showing results for 
Search instead for 
Do you mean 
 

Build a Location based Search Engine with Haven OnDemand

Using HPE Haven OnDemand Text Indexing and Search APIs you can quickly and easily enhance the functionality of any app by implementing an advanced search engine that is capable of searching by natural language text, keywords and geo location without the need to write any algorithms.

In this blog, I will show you how to use Haven OnDemand APIs to build a custom searchable database, as well as a web-application to search for information based on geo-location.

If you are new to Haven OnDemand, please click here to create an account. Once your account is setup, click here to copy the API key so you can use it to call the Haven OnDemand APIs.

To keep this blog easy to read, I will skip discussions about how to add, initialize and implement Google maps API. Also, the demo code in this blog is just for illustration and might not be reliable for copy/paste. If you need the complete code, click here to download the entire project’s source.

Example use case

To make it easier to follow along, we’ll use a use case to help showcase one possible use of this solution. Of course this is not the only use case for this solution, and we are only limited by our imaginations.

For this use case, assume that you have written a story about the Golden Gate Bridge, called it a POI (Point of Interest) and published it on your web site. You may want it to be searchable not only by keyword references and synonyms, but also by the precise location of where the Golden Gate Bridge is situated. To search by precise location, you must provide the “address” of the Golden Gate Bridge. Well, there is no street number for such a place. Instead, you need to use the map coordinates with the longitude and latitude values of the POI.

The accuracy of a POI will be defined by the numbers of the decimal number in the longitude and latitude values. Therefore, it is impractical to search for a POI by comparing the exact longitude and latitude numbers. To find a POI by its map coordinates, we will need some algorithm to calculate the whereabouts of the POI within an area defined by a radius. We could either write an algorithm to determine this the hard way, or we could simply use the Haven OnDemand APIs to save a lot of time, effort and frustration.

Create a Text Index with Haven OnDemand

The first thing we need is a text index where we can store content and metadata for our search engine.

Let’s login to Haven OnDemand and move the mouse to the top-right corner where we find a person icon and select the Account option from the dropdown list. At the Account page, click the Text Indexes button under the Services menu then click the Create text index button.

pic1.png

At the Create Text Index form, specify a name. In this tutorial, we’ll call it “havensearch”. Optionally provide the Display name and the Description if you want to.

Click the Next button and choose “Custom Fields” from the Flavor list.

Click the Next button and enter the following information onto the form:

  • Enter category into the Parametric Fields field
  • Scroll down and enter lon space lat into the Numeric field.

Click Next, then Create button to complete the text index creation.

Alternatively, you can create a text index by calling the Create Text Index API from your code. See the last section (Create Text Index Using Haven OnDemand PHP library) of this blog for details.

 

Build a Web app

We will create a simple web app with 2 pages - one page (create.html) for adding content to our search database and another page (index.html) for searching content from our database. On both pages, we will use Google maps API to display a Google map where we can click on a location to get the longitude and latitude of that point, which we will call a POI.

Let’s now create a create.html page and add the form below to the page.

 

<form id="dataform" method="post">
    <span>Latitude: </span>
    <input id="lat" name="lat" type="number" value="" size=13/>
    <span>Longitude: </span>
    <input id="lon" name="lon" type="number" value="" size=13/>
    <span>Select a category</span><br/>
    <select name="cat">
        <option selected value="attraction">Attraction</option>
        <option value="recreation">Recreation</option>
        <option value="museum">Museum</option>
        <option value="business">Business</option>
    </select><br/><br/>
    <span>Enter document link</span><br/>
    <input name="ref" type="text" value="" size=30/><br/><br/>
    <input type="button" value="Submit" onclick="sendRequest()">
</form>
<div id="result"></div>

<script type="text/javascript">
    function sendRequest() {
  var endpoint = "create.php";
        $.post(endpoint, $("#dataform").serialize(), requestCallback);
    }
    function requestCallback(data) {
        $("#result").html(data);
    }
</script>

From the code above, we have 2 inputs to hold the longitude and latitude of a point, a selection to select a predefined category, and an input for specifying the link to the article. We also implement a JavaScript function called sendRequest() and use jQuery to send a post request to our web server. When the response arrives, we will output the data from the requestCallback function.

Add Data to a Text Index

We use PHP programming language to write a server app called “create.php”, which can call the Add to Text Index API to add data to the “havensearch” text index.

class POI {
    public $reference = "";
    public $category = "";
    public $lat = 0;
    public $lon = 0;
}
$newPOI = new POI();
$newPOI->reference = $_REQUEST['ref'];
$newPOI->lat = $_REQUEST['lat'];
$newPOI->lon = $_REQUEST['lon'];
$newPOI->category = $_REQUEST['cat'];

We start by defining a data class named POI with 4 data members. Every time we receive a request from the client app, we create a $newPOI object of the POI class, parse the request data, and assign the value to the object’s data members accordingly. Then, we serialize the POI object to a JSON string and we specify the parameters for the Add to Text Index API call as shown below:

$json = json_encode($newPOI);
$paramArr = array(
    'index' => "havensearch",
    'url' => $_REQUEST['ref'],
    'additional_metadata' => $json
);

For the parameters specified above, we tell Haven OnDemand to fetch the content and metadata of a document from the URL provided in the $_REQUEST['ref'] and then add the data to our text index database name “havensearch”. With the additional_metadata specified, we ask the Add to Text Index API to use the provided URL as the item’s reference ID inside the text index and also add the latitude, longitude, and category values to the text index fields accordingly.

For convenience, we will use the Haven OnDemand library for PHP so we can easily call Haven OnDemand APIs. Follow the instructions on the README to get the library.

We then create an instance of the HODClient() and call the PostRequest to send the data to Haven OnDemand.

 

$client = new HODClient("HOD-APIKEY");
$hodApp = HODApps::ADD_TO_TEXT_INDEX;
$client->PostRequest($paramArr, $hodApp, REQ_MODE::SYNC, 'requestCompleted');

IMPORTANT: The reason why we want to call Haven OnDemand APIs from our PHP script is that we don’t want to put the secret API key in our JavaScript code.

 

Let’s assume the API call is successfully. We get the response to the callback function requestCompleted, where we will echo the response to the client web app.

 

function requestCompleted($data) {
    $resp = new HODResponseParser($data);
    if ($resp->status == "finished") {
        echo json_encode$resp->payloadObj);
    }
}

If the data was added successfully to the “havensearch” index, the response will be returned to our create.html app and it will look like this:

{"index":"havensearch","references":[{"reference":"http:\/\/www.somedomain.com","id":1}]}

That is all we need to implement for making a location based document to be searchable by its geo location.

Query the Text Index

Now it’s time to implement a search web app that can search for information based on geo location.

We create an index.html page and add the form below to the page.

<form id="dataform" method="post">
    <span>Enter radius (in km)</span>
    <input id="rad" name="rad" type="number" value="5" onchange="newRad()"/>
    <span>Select a category</span>
    <select id="cat" name="cat" onchange="sendRequest()">
        <option value="attraction">Attraction</option>
        <option value="recreation">Recreation</option>
        <option value="museum">Museum</option>
        <option value="business">Business</option>
        <option selected value="all">All</option>
    </select>
    <input id="lat" name="lat" type="number" hidden />
    <input id="lon" name="lon" type="number" hidden />
</form>

<script type="text/javascript">
function sendRequest() {
    var endpoint = "search.php";
    $.post(endpoint, $("#dataform").serialize(), requestCallback);
}
function requestCallback(data) {
    // parse response
}
</script>

In this demo app, we have an input field rad to get user’s defined radius and a dropdown list cat to get user’s selected category. We also use Google maps API to display a map and detect a mouse click to get the latitude and longitude of a location. Every time a user clicks on the map or change the radius or change the category selection, we will end up in calling the sendRequest function, which will send the data to our PHP app named “search.php”.

 

Let’s move on to write a server app called “search.php”, which will receive the client request and call the Query Text Index API to search for content from the “havensearch” text index.

First of all, we define a fieldtext variable and specify a geographical search syntax using the DISTPHERICAL operator and set the latitude, longitude and radius as shown below.

 

$fieldtext = "(DISTSPHERICAL{".$_REQUEST["lat"].",".$_REQUEST["lon"].",".$_REQUEST["rad"]." in KM}:lat:lon)";

Then, we detect if the user has chosen a particular category. If so, we use the AND and the MATCH operators to set the category and append it to the fieldtext variable as shown below.

 

if ($_REQUEST['cat'] != "all")
    $fieldtext .= " AND (MATCH{".$_REQUEST['cat']."}:category)";

Finally, we specify the parameters for the Query Text Index API call.

 

$paramArr = array(
    'indexes' => "havensearch",
    'text' => "*",
    'field_text' => $fieldtext,
    'summary' => "quick",
    'print_fields' => "category,lat,lon",
    'absolute_max_results' => 40
    );

With the parameters specified above, we ask the Query Text Index API to search for everything (with the text specified as an asterisk “*”) from the “havensearch” text index database. The search result must satisfy all the conditions specified by the field_text paremter. The response should include the summary of an item’s content together with the value of the category, lat and lon index fields. Additionally, we want to have at maximum 40 items per request.

 

We will also use the Haven OnDemand library for PHP to call Haven OnDemand APIs.

$client = new HODClient("HOD-APIKEY");
$hodApp = HODApps::QUERY_TEXT_INDEX;
$client->PostRequest($paramArr, $hodApp, REQ_MODE::SYNC, 'requestCompleted');

Let’s assume that the API call is successfully then we get the response to the callback function requestCompleted, where we will echo the response to the client Web app.

function requestCompleted($data) {
    $resp = new HODResponseParser($data);
    if ($resp->status == "finished") {
        echo json_encode($resp->payloadObj);
    }
}

Now we come back to the index.html file and complete the requestCallback function.

var POIs = [];
function requestCallback(response) {
    jsonObj = JSON.parse(response);
    for (var i = 0; i < jsonObj.documents.length; i++) {
        var doc = jsonObj.documents[i];
        var POI = new google.maps.Marker({
            position: new google.maps.LatLng(doc.lat[0],doc.lon[0]),
            map: map,
            draggable: false,
            title: doc.summary + ".\nClick to read more.",
            link: doc.reference
        });
        if (doc.wikipedia_type[0] == "attraction") {
            POI.setIcon('http://maps.google.com/mapfiles/ms/icons/blue-dot.png')
        } else if (doc.wikipedia_type[0] == "recreation") {
            POI.setIcon('http://maps.google.com/mapfiles/ms/icons/green-dot.png')
        } else if (doc.wikipedia_type[0] == "museum"){
            POI.setIcon('http://maps.google.com/mapfiles/ms/icons/red-dot.png')
        } else {
            POI.setIcon('http://maps.google.com/mapfiles/ms/icons/green-dot.png')
        }
        google.maps.event.addListener(poi, 'mouseover', function(ev) {
            this.getMap().getDiv().setAttribute('title',this.get('title'));
        });
        google.maps.event.addListener(poi,'mouseout',function(){
            this.getMap().getDiv().removeAttribute('title');
        });
        google.maps.event.addListener(poi,'click',function(){
            var link = this.get('link');
            window.open(link);
        });
        POIs.push(POI);
    }
}

The response from Haven OnDemand Query Text Index is a JSON string, so we call JavaScript JSON.parse(response) function convert the response into a JSON object. Note that the response has the documents as a JSON array, we detect the length of the array and loop through the array to access its item values.

 

For each found item, we create a POI using google.maps.Marker and assign the value of that item as shown in the code above.

 

We also add some event listeners to the Google maps for each POI so that when the mouse is hovered over the POI, we show the summary of that POI. When a user clicks on a POI, we will open a new tab in the web browser with the URL taken from the item’s reference value.

Finally, we add the newly created POI to the POIs array. The reason we keep an array of POI is just for clearing the old POIs whenever we search for new documents.

 

That is all we need for our app. Let’s copy all the files (create.html, index.html, create.php and search.php) to a web server and open the create.html page from a web browser.

 

  • Type “San Francisco” to the input field above the map and click the Go button. Or, simply click to drag the map to San Francisco area.
  • Click on any part of the Golden Gate Bridge to get the latitude and longitude. The values will be automatically added to the latitude and longitude inputs of the form on the right-hand side.

pic2.png

  • Select the Attraction for the category.
  • We will use a Wikipedia page as the source of our information, so add this link to the document link input field https://en.wikipedia.org/wiki/Golden_Gate_Bridgepic3.png
  • Click the Submit button and wait for the response to arrive.
  • Open a new tab on the Web Browser and open the index.html page.
  • Type “San Francisco” to the input field above the map and click the Go button. Or, simply click to drag the map to San Francisco area.
  • Click anywhere near the Golden Gate Bridge and you will see a circle with new POI at the location where you clicked when making that POI.

pic4.png

 

Create Text Index Using Haven OnDemand PHP library

 

$paramArr = array(
    'index' => "havensearch",
    'description' => "advanced search database",
    'display_name' => "My demo search index",
    'flavor' => "custom_fields",
    'numeric_fields' => ["lat","lon"],
    'parametric_fields' => ["category"]
);
$client = new HODClient("HOD-APIKEY");
$hodApp = HODApps::CREATE_TEXT_INDEX;
$client->PostRequest($paramArr, $hodApp, REQ_MODE::SYNC, 'requestCompleted');

 

As I mentioned in the “Add Data to the Text Index” section, the content of the document from the provided URL is automatically indexed. That means we can also provide enhanced search by keywords or by phrases together with other field text operators. I will leave that unexploited so you can discover yourself by further reading the capability of Haven OnDemand Query Text Index.

 

Phong Vu

Social Media
About the Author
Topics
† The opinions expressed above are the personal opinions of the authors, not of HPE. By using this site, you accept the Terms of Use and Rules of Participation