Search our Blogs
Showing results for 
Search instead for 
Do you mean 
 

Batch jobs using Haven OnDemand's APIs

Haven OnDemand allows you to batch multiple API jobs in a single request using the Job API, for example, to analyze a batch of web pages, documents, or social media messages where you need to analyze each text individually but want to be more efficient with your code, or where you want to execute multiple API calls on a single web page, document, or text.

 

The Job API accepts two parameters - ‘apikey’ and ‘job’, which are, respectively, your API key and a JSON object that specifies the APIs you are calling (their endpoints), the version of the API, and the parameters you are passing to them. Furthermore, the JSON object will contain an array called 'actions' which specifies the aforementioned. Here is an example of the structure of the ‘job’ JSON object:

 

 

{
  "actions": [
     { "name": "analyzesentiment",
        "version": "v1",
        "params": {
           "text": "I love dogs"
        }
     },
     { "name": "extractconcepts",
        "version": "v1",
        "params": {
           "text": "I love dogs"
        }
     }
  ]
}

 

Example

In this example, we will do the following:

 

Rather than making multiple, separate API calls to Text Extraction, Sentiment Analysis, and Concept Extraction, we can batch them together into a single API request. Note: you don’t have to use the same text, file, or URL for each job in the Job API. You can mix and match with parameters that have no relation.

 

 

curl -X POST http://api.havenondemand.com/1/job \
-H "Content-Type:multipart/form-data" \
-F "apikey=<YOUR_APIKEY>" \
-F "job={\"actions\":[{\"name\":\"extracttext\",\"version\":\"v1\",\"params\":{\"file\":\"file1\"}},{\"name\":\"analyzesentiment\",\"version\":\"v1\",\"params\":{\"text\":\"Hey, good job with that thing you did recently\"}},{\"name\":\"extractconcepts\",\"version\":\"v1\",\"params\":{\"url\":\"http://en.wikipedia.org/wiki/United_Kingdom\"}}]};type=application/json" \
-F "file1=@IDOLbrochure.pdf"

 

This is an asynchronous request, which will return back a job ID.

 

{"jobID": "<JOB_ID>"}

 

To see the results, pass this job ID into either the Status or Result API. It is recommended to use the Status API as this will not timeout when checking. This will tell you if the job has failed, is still processing, or is complete with the results.

 

curl -X GET https://api.havenondemand.com/1/job/status/<JOB_ID>?apikey=<YOUR_APIKEY>

 

In our example, the response delivers the results:

 

 

{
   "actions": [
       {
           "result": {
               "document": [
                   {
                       "reference": "IDOLbrochure.pdf",
                       "doc_iod_reference": "ab5afe5a3639209c56fc60c12011d51a",
                       "app_name": [
                           "Adobe PDF Library 9.9"
                       ],
                       "author": [
                           "HP Autonomy"
                       ],
                       "content_type": [
                           "application/pdf"
                       ],
                       "created_date": [
                           "1378283173"
                       ],
                       "creator": [
                           "Adobe InDesign CS5.5 (7.5)"
                       ],
                       "document_attributes": [
                           "0"
                       ],
                       "document_embedded_font_ratio": [
                           "21"
                       ],
                       "document_pct_embedded_font": [
                           "100"
                       ],
                       "document_pct_probability_mismatch": [
                           "21"
                       ],
                       "file_size": [
                           637147
                       ],
                       "modified_date": [
                           1397519305
                       ],
                       "page_count": [
                           "4"
                       ],
                       "processing_error_code": [
                           "1"
                       ],
                       "processing_error_description": [
                           "Document: Embedded Font"
                       ],
                       "subject": [
                           "HP IDOL 10: built for the era of big data"
                       ],
                       "title": "Next-generation information analytics",
                       "content": "Brochure Next-generation information analytics HP IDOL 10: built for the era of big data Get your greatest return on information We are experiencing transformational changes in the computing arena. Data is doubling every 12 to 18 months, accelerating the pace of innovation and time-to-value. The vast majority of today’s data growth is driven by the creation of human-friendly information such as documents, social content, video, audio, and images. In fact, 90% of digital content being created will include these unstructured data types by 20151 . Data growth is also accelerated by the ubiquity of mobile devices, which gives us “always on” applications and users who demand anytime, anywhere access to information. As a business leader, you depend on human information to keep your business working to its full potential. The information analytics platform, HP IDOL 10 (Intelligent Data Operating Layer), from HP Autonomy, has been built to address these modern challenges. IDOL 10 offers you a single processing layer that helps you automatically unlock key ideas and concepts in all your information, structured and unstructured. Based on advanced mathematical principles and augmented by over 170 patents, IDOL 10 understand the key concepts present in virtually all forms of content, helping you to detect patterns and relationships between data. With IDOL 10, you can understand and act upon documents, emails, video, chat, phone calls, social media, and application data at the same time and faster than ever before. Because we know information is fragmented across a variety of repositories, IDOL 10 streamlines information processing across networks, the web, and the cloud. We help you see your information wherever it is. After all, why should you work for your information when it can work for you? IDOL 10 powers many of the applications that depend upon an understanding of human information to perform, including: • Enterprise Search • eDiscovery • Voice of the Customer/Workforce • Marketing Optimization • Enterprise Content Management • Social Media Analytics Brochure | Next-generation information analytics Highlights • Understand virtually all of your information with high-performance analytics: Over 500 analytical functions available for text, audio, video, and image • Derive actionable insights: Process data in near real time to gain a competitive edge • Maximize your information reach: Connect to over 400 systems with support for over 1000 file formats, so you can find all relevant information • Let social media work for you: Detect emerging trends and influencers in this powerful media with sophisticated sentiment analysis and clustering technology • Achieve big data scalability and security: Track record of supporting the largest enterprises in volume of information, number of users, and frequency of transactions while adhering to strict security standards • Easy to use and administer: Leverage a visual dashboard and simplified processes • Manage data in place: Manage data within the original repository while ensuring its accessibility • Rapid innovation: Roadmap includes additional next-gen capabilities via quarterly releases IDOL 10 understands virtually all information fragmented across different silos, and finds patterns and relationships surfaced through different applications 1 IDC Predictions 2012: Competing for 2020 Brochure | Next-generation information analytics If you can’t find your data, does it really exist? We have realized that traditional ways of understanding and using information simply fall short in providing the business benefits that are required in today’s world. When detectives try to uncover a crime, they look for incriminating emails and wade through surveillance videos. When marketers want to know more about their customers, they look to Twitter. Information today is diverse, dynamic, and complicated. The old method of finding answers or uncovering predictive patterns in the rows and columns of a database take far too much time, leave out gaps of knowledge, and often yield incomplete results. Similarly, keyword and metadata-based search return too many irrelevant and incomplete results, without giving actionable insights to make the information count. The explosion in the variety of file formats leaves most search and analytics technologies incapable of making sense of new file types. Since they rely on user-defined tags, which are often inconsistent and too vague to provide much meaning, you inevitably can’t connect with information you probably have, but for all practical purposes don’t know exists. There is simply too much data to sift through all at once without using a system that can actually understand it, and this unfortunately renders a lot of great information unsearchable and unusable. Understanding meaning helps you unleash the value of your information When you can analyze all types of information at once, you can make decisions while there is still time to change your course. For instance, what if you needed to simultaneously analyze all customers in one zip code who use an iPhone 4, provided product feedback when they called customer service, and posted a Yelp review in the same month? If you processed the data manually, it would take so long that the information would lose its value before you could use it. With IDOL 10, you can now perform sophisticated analytics on all types of information at the same time, and faster than ever before. IDOL 10 uses patented probabilistic and pattern-matching algorithms to automatically recognize concepts and ideas in all your information. Because the technology is based on the universal language of mathematics, it treats words as “symbols” and works in all languages. IDOL 10’s unique technology detects patterns, emotions, sentiments, intentions, and preferences as they happen. Knowing more about your data in less time speeds the pace of business and strengthens your ability to compete. “Recruiters are highly skilled individuals, and that’s not something you can replace—but what we can do is give them a tool that helps them work more efficiently. With IDOL, instead of 50 loosely-related candidates to review, recruiters get the 15 most suitable, and work from there.” —Daniel Richardson, CTO, Bond International Software IDOL 10 provides visualization tools to reveal insights about data “HP Autonomy, with the power of its IDOL engine, takes fan data, collects it, and stitches it together…that helps us understand what is being talked about across the ecosystem of the sport.” —Steve Worling, Senior Director of IT, NASCAR Brochure | Next-generation information analytics About HP Autonomy HP Autonomy is a global leader in software that processes human information, or unstructured data, including social media, email, video, audio, text, web pages, and more. Autonomy’s powerful management and analytic tools for structured information together with its ability to extract meaning in real time from all forms of information, regardless of format, is a powerful tool for companies seeking to get the most out of their data. Autonomy’s product portfolio helps power companies through enterprise search analytics, business process management and OEM operations. Autonomy also offers information governance solutions in areas such as eDiscovery, content management and compliance, as well as marketing solutions that help companies grow revenue, such as web content management, online marketing optimization and rich media management. Please visit autonomy.com to find out more. Learn more at autonomy.com/idol10 Share with colleagues Sign up for updates hp.com/go/getupdated © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Trademark acknowledgements if needed. 20130902_PI_B_HP_AUTN_IDOL10"
                   }
               ],
               "md5sum": [
                   "5c9e969ec9ae95fba93942ab995fc2d6"
               ]
           },
           "status": "finished",
           "action": "extracttext",
           "version": "v1"
       },
       {
           "result": {
               "positive": [
                   {
                       "sentiment": "good",
                       "topic": "job with that thing",
                       "score": 0.6349022358073864,
                       "original_text": "good job with that thing",
                       "original_length": 24,
                       "normalized_text": "good job with that thing",
                       "normalized_length": 24
                   }
               ],
               "negative": [],
               "aggregate": {
                   "sentiment": "positive",
                   "score": 0.6349022358073864
               }
           },
           "status": "finished",
           "action": "analyzesentiment",
           "version": "v1"
       },
       {
           "result": {
               "concepts": [
                   {
                       "concept": "United Kingdom",
                       "occurrences": 584
                   },
                   {
                       "concept": "Ireland",
                       "occurrences": 393
                   },
                   {
                       "concept": "Northern Ireland",
                       "occurrences": 193
                   },
                   {
                       "concept": "Great Britain",
                       "occurrences": 93
                   },
                   {
                       "concept": "London",
                       "occurrences": 145
                   },
                   {
                       "concept": "Irish",
                       "occurrences": 143
                   },
                   {
                       "concept": "Islands",
                       "occurrences": 212
                   },
                   {
                       "concept": "Scottish",
                       "occurrences": 113
                   },
                   {
                       "concept": "UK",
                       "occurrences": 311
                   },
                   {
                       "concept": "European Union",
                       "occurrences": 50
                   },
                   {
                       "concept": "Welsh",
                       "occurrences": 87
                   },
                   {
                       "concept": "Office",
                       "occurrences": 100
                   },
                   {
                       "concept": "Royal",
                       "occurrences": 57
                   },
                   {
                       "concept": "Wikipedia",
                       "occurrences": 45
                   },
                   {
                       "concept": "Scottish Gaelic",
                       "occurrences": 22
                   },
                   {
                       "concept": "Scots",
                       "occurrences": 50
                   },
                   {
                       "concept": "Crown",
                       "occurrences": 39
                   },
                   {
                       "concept": "Welsh language",
                       "occurrences": 17
                   },
                   {
                       "concept": "West",
                       "occurrences": 58
                   },
                   {
                       "concept": "Ulster",
                       "occurrences": 27
                   },
                   {
                       "concept": "Irish Free",
                       "occurrences": 14
                   },
                   {
                       "concept": "Search",
                       "occurrences": 27
                   },
                   {
                       "concept": "countries",
                       "occurrences": 143
                   },
                   {
                       "concept": "City",
                       "occurrences": 32
                   },
                   {
                       "concept": "Island",
                       "occurrences": 33
                   },
                   {
                       "concept": "Queen",
                       "occurrences": 19
                   },
                   {
                       "concept": "Flag",
                       "occurrences": 27
                   },
                   {
                       "concept": "largest",
                       "occurrences": 70
                   },
                   {
                       "concept": "Cornish",
                       "occurrences": 14
                   },
                   {
                       "concept": "See",
                       "occurrences": 20
                   },
                   {
                       "concept": "article",
                       "occurrences": 38
                   },
                   {
                       "concept": "continental Europe",
                       "occurrences": 6
                   },
                   {
                       "concept": "God Save",
                       "occurrences": 6
                   },
                   {
                       "concept": "White",
                       "occurrences": 20
                   },
                   {
                       "concept": "Royal coat",
                       "occurrences": 5
                   },
                   {
                       "concept": "North East",
                       "occurrences": 5
                   },
                   {
                       "concept": "arms",
                       "occurrences": 27
                   },
                   {
                       "concept": "containing",
                       "occurrences": 30
                   },
                   {
                       "concept": "page",
                       "occurrences": 42
                   },
                   {
                       "concept": "two",
                       "occurrences": 28
                   }
               ]
           },
           "status": "finished",
           "action": "extractconcepts",
           "version": "v1"
       }
   ],
   "jobID": "<JOB_ID>",
   "status": "finished"
}

 

 

Most Haven OnDemand APIs support using the Job API, so you can create a variety of jobs when batching together.

Tags: Job API
Comments
jinumm
| ‎07-07-2016 10:32

thank you for the detailed api codes , i need this for my merchant service provider in usa , any way thank you very much 

Social Media
About the Author
Topics
† The opinions expressed above are the personal opinions of the authors, not of HPE. By using this site, you accept the Terms of Use and Rules of Participation