Navigate to the Rule page as shown in the screenshot below. By selecting the icon on the top left corner under the Owl icon and select “Rule”. You will see a page as shown below.
You can get to rules from the hoot page by clicking on Rules under the dataset name and score in the top right of the screen.
Search for a dataset
Rules can only be applied to datasets that have had an Owlcheck scan or previously catalogued
This will populate the schema and any previously saved rules
Select a rule type
Using the dropdown next to the Type label
Select a rule name
If applying a preset rule, the rule name will be auto populated
Input a rule condition
Only if applying a simple, freeform sql, or function rule type, provide a value in the condition/sql/function input field.
Keystroke Ctrl+Space will provide IntelliSense
Select Low, Medium or High scoring sever
Click submit to save the rule.
The rule will be applied to the next OwlCheck run on that particular dataset.
When to use a simple rule:
Simple rules would be applied to filter a condition on a single column in a single table.
Example: city = 'Baltimore'
When to use a freeform sql rule:
Complex (freeform sql) would be used when applying a condition across multiple tables/columns and generally when more flexibility/customization is desired.
Example: select * from dataset where name = 'Owl'
When to use a preset rule:
Preset rules would be used for quickly adding strict condition check. Commonly used conditions are available to add to any dataset columns.
All built-in spark functions are available to use. (https://spark.apache.org/docs/2.3.0/api/sql/) for simple and freeform sql rules.
For every percentage the X condition occurs, deduct Y points from the data quality score. If a rule was triggered 10 times out of 100 rows, break records occurred 10% of the time. If you input 1 point for every 1 percent, 10 points would be deducted from the overall score.
Let’s create a simple rule using the below information. The dataset name.
Search for “shape_example” and click “Load”
Select “Simple Rule”
Rule Name = lnametest
@shape_example.lname = “hootbeck” (should hit one time day over day).
Points = 1
Percentage = 1
Once the rule has been submitted please find the below list of rules with the new rule we just defined as shown below.
Rule scores will appear under the Rule tab on the Hoot page. You can also see more details in the bottom panel of the Rules page under the Rules and Results tabs.