Today we continue discussing the best practice from storage engineering:
233) Strategies never remain the same if the data and the business change. Consequently, even the longest running strategy is constantly re-evaluated to see if it can still perform as well. This has in fact been demonstrated in commercial database systems with the use of query compilation and recompilation and holds equally true for classifiers and other kinds of analysis.
234) Since strategy is best described by logic, it is very helpful to export is as a module so that it can run anywhere after being written once. This has been demonstrated by my machine learning packages and data mining algorithms regardless of the domain in which the data exists. At a low-level, the same applies to strategies within individual components because even if they are not immediately re-used, it will be helpful to have version control on them.
235) Optimization of an execution does not merely depend on the data and the strategy. It involves hints from the users, environmental factors and parameters. All of this play a role in driving down the costs and some are easier to tweak than others.
236) Builtins from storage products are helpful to customers because they are already available and tested. However the use of builtins may not always be deterministic and free of side-effects. Care must be taken to document how the behavior might change.
237) Customers might not see how a speed up may occur in complex systems. While auto-tuning may cover some scenarios, the bulk of scenarios at user level maybe covered with automation.
238) Parametrization helps queries because the plans remains the same. Even if the storage
product merely implements API for listing resources, it could parametrize the filters to apply and the same may be applied to all other similar API.
#codingexercise
Int GetSortedTail(String a) {
For (int i = a.length - 1; i >=0; i--)
For (int j = a+1; j < a.length; j++)
{
If (a[i] > a[j]) {
Return len - i;
}
}
Return len;
}
233) Strategies never remain the same if the data and the business change. Consequently, even the longest running strategy is constantly re-evaluated to see if it can still perform as well. This has in fact been demonstrated in commercial database systems with the use of query compilation and recompilation and holds equally true for classifiers and other kinds of analysis.
234) Since strategy is best described by logic, it is very helpful to export is as a module so that it can run anywhere after being written once. This has been demonstrated by my machine learning packages and data mining algorithms regardless of the domain in which the data exists. At a low-level, the same applies to strategies within individual components because even if they are not immediately re-used, it will be helpful to have version control on them.
235) Optimization of an execution does not merely depend on the data and the strategy. It involves hints from the users, environmental factors and parameters. All of this play a role in driving down the costs and some are easier to tweak than others.
236) Builtins from storage products are helpful to customers because they are already available and tested. However the use of builtins may not always be deterministic and free of side-effects. Care must be taken to document how the behavior might change.
237) Customers might not see how a speed up may occur in complex systems. While auto-tuning may cover some scenarios, the bulk of scenarios at user level maybe covered with automation.
238) Parametrization helps queries because the plans remains the same. Even if the storage
product merely implements API for listing resources, it could parametrize the filters to apply and the same may be applied to all other similar API.
#codingexercise
Int GetSortedTail(String a) {
For (int i = a.length - 1; i >=0; i--)
For (int j = a+1; j < a.length; j++)
{
If (a[i] > a[j]) {
Return len - i;
}
}
Return len;
}
No comments:
Post a Comment