Wednesday, February 17, 2021

User interface and API design for Streaming Queries (continued from previous post ...)

 Analytical applications that perform queries on data are best visualized via catchy charts and graphs. Most applications using storage and analytical products have a user interface that lets them create containers for their data but have very little leverage for the customer to author streaming queries. Even the option to generate a query is nothing more than the uploading of a program using a compiled language artifact. The application reuses a template to make requests and responses, but this is very different from long running and streaming responses that are typical for these queries. A solution is presented in this document. 

Query Interface is not about keywords. 

When it comes to a user interface for querying data, people often associate the Google user interface with search terms. Avoid Google. This is not necessarily the right interface for Streaming queries. If anything, it is simplistic, and keywords based. It is not operators-based. Splunk interface is a much cleaner interface since it has seasoned from search over machine data. Streaming queries are not table queries. They also don’t operate on big data on the Hadoop file system only. They do have standard operators that can directly translate to operator keywords and can be chained the same manner in a java file as pipe operators on the search bar. These translations of query operator expressions on the search bar to a java language FLink streaming query is a patentable design for streaming user interface and one that will prove immensely flexible for direct execution of user queries as compared to package queries. 


Use of off-the-shelf JavaScript libraries for data visualization. 

 

There are several off-the-shelf libraries for data visualization such as D3.js, Recharts, and other libraries including admin-UI for dashboards. They are used on a case-by-case basis. User-interface for a dedicated purpose requires a lot of customization. Sometimes it is easy to work with off-the-shelf-libraries, other times, it is simpler to rewrite the foundation for the essential and long-standing use-cases.  

 

Conclusion: 

User interface development for streaming queries should bring the best practice of usability from those for objects store and database while bringing the best out of the event-by-event continuity that is the hallmark of stream processing. Paginated results are the hallmark of querying over big data whether the more relevant results are brought up first even if all the results follow later. Results from streaming queries can also be shown as they arrive so that the users can have a chance to pause if they have what they need already. They will also benefit from piped operators that can be entered in the search bar just the same way as keywords and this permits the results to be extracted-transformed-and-loaded to forms that appeal better to the user. 


No comments:

Post a Comment