Sunday, February 11, 2018


We were looking at some of the search queries that are collected from the community of those using  logs from an identity provider:

Some other interesting events for identity include:

45) looking for a few lines above and below a match to include associated event attributes. This is easy with a streaming operation in the shell command with "grep –C<N> literal file".  In SQL this becomes slightly complicated involving a recursive common table expression. A nested query might work too provided the identifiers are continuous.
For example:
SELECT a.*
FROM Table1 as a,
(SELECT id FROM Table1 WHERE message LIKE '%hello%') as b
WHERE a.ids BETWEEN b.id-N AND b.id+N;
On the other hand by using max(b.id) < id  and  min(b.id) > id as the sentinels, we can now advance the sentinels row by row in a recursive query to always include a determined number of lines above and below the match

46) grouping selections and counting now works successfully with the above logic. For example, if we are searching for http requests in a long that span multiple lines one for each request parameter, then we could include the associated parameters to corresponding to the requests that match as tags to group the requests. For example

grep -C7 match file | grep tag | cut -d"=" -f1 | sort | uniq -c | sort -nr

47) In the absence of an already existing tags, we can now create new tags with search and replace command in the same logic as above but with piping operation..

No comments:

Post a Comment