tsio/update: inhibit the formula patching capability
test/operator/add: document odd corner case
test/api: this test better at the api level
This tshistory component provides a formula language to build computed series.
Formulas are defined using a simple lisp-like syntax, using a pre-defined function library.
Formulas are read-only series (you can't update
or replace
values).
They also have an history, which is built, time stamps wise, using the union of all constituent time stamps, and value wise, by applying the formula.
Because of this the staircase
operator is available on formulae.
Some staircase
operations can have a very fast implementation if the
formula obeys commutativity rules.
Formulas are expressed in a lisp-like syntax using operators
,
positional (mandatory) parameters and keyword (optional) parameters.
The general form is:
(<operator> <param1> ... <paramN> #:<keyword1> <value1> ... #:<keywordN> <valueN>)
Here are a couple examples:
(add (series "wallonie") (series "bruxelles") (series "flandres"))
Here we see the two fundamental add
and series
operators at work.
This would form a new synthetic series out of three base series (which can be either raw series or formulas themselves).
Some notes:
operator names can contain dashes or arbitrary caracters
literal values can be: 3
(integer), 5.2
(float), "hello"
(string) and #t
or #f
(true ot false)
Performs a scalar product on a series.
Example: (* -1 (series "positive-things"))
Add a constant quantity to a series.
Example: (+ 42 (series "i-feel-undervalued"))
Perform a scalar division between numbers or a series and a scalar.
Example: (/ (series "div-me") (/ 3 2))
Linear combination of two or more series. Takes a variable number of series as input.
Example: (add (series "wallonie") (series "bruxelles") (series "flandres"))
To specify the behaviour of the add
operation in the face of missing
data, the series can be built with the fill
keyword. This option is
only really applied when several series are combined. By default, if
an input series has missing values for a given time stamp, the
resulting series has no value for this timestamp (unless a fill rule
is provided).
Set an upper/lower threashold for a series. Takes a series as
positional parameter and accepts two optional keywords min
and max
which must be numbers (integers or floats).
Example: (clip (series "must-be-positive") #:min 0)
Produces an utc timestamp from its input string date in iso format.
The tz
keyword allows to specify an alternate time zone.
The naive
keyword forces production of a naive timestamp.
Both tz
and naive
keywords are mutually exlcusive.
Element wise division of two series.
Example: (div (series "$-to-€") (series "€-to-£"))
Computes the row-wise minimum of its input series.
Example: (min (series "station0") (series "station1") (series "station2"))
Computes the row-wise maximum of its input series.
Example: (max (series "station0") (series "station1") (series "station2"))
Element wise multiplication of series. Takes a variable number of series as input.
Example: (mul (series "banana-spot-price ($)") (series "$-to-€" #:fill 'ffill'))
This might convert a series priced in dollars to a series priced in euros, using a currency exchange rate series with a forward-fill option.
Allow demoting a series from a tz-aware index (strongly recommended) to a tz-naive index (unfortunately sometimes unavoidable for interop with other tz-naive series).
One must provide a country code and a target timezone.
Example: (naive (series "tz-aware-series-from-poland") "PL" "Europe/Warsaw")
The priority operator combines its input series as layers. For each timestamp in the union of all series time stamps, the value comes from the first series that provides a value.
Example: (priority (series "realized") (series "nominated") (series "forecasted"))
Here realized
values show up first, and any missing values come from
nominated
first and then only from forecasted
.
Resamples its input series using freq
and the aggregation method
method
(as described in the pandas documentation).
Example: (resample (series "hourly") "D")
This operator computes the row-wise mean of its input series using the
series weight
option if present. The missing points are handled as
if the whole series were absent.
Example: (row-mean (series "station0") (series "station1" #:weight 2) (series "station2"))
Weights are provided as a keyword to series
. No weight is
interpreted as 1.
The series
operator accepts several keywords:
fill
to specify a filling policy to avoid nans
when the series
will be add
ed with others; accepted values are "ffill"
(forward-fill), "bfill"
(backward-fill) or any floating value.For instance in (add (series "a" #:fill 0) (series "b")
will make
sure that series a
, if shorter than series b
will get zeroes
instead of nans where b
provides values.
This allows cutting a series at date points. It takes one positional
parameter (the series) and two optional keywords fromdate
and
todate
which must be strings in the iso8601 format.
Example: (slice (series "cut-me") #:fromdate "2018-01-01")
Computes the standard deviation over its input series.
Example: (std (series "station0") (series "station1") (series "station2"))
Takes a timestamp and a number of years, months, weekds, days, hours, minutes (int) and computes a new date according to the asked delta elements.
Example: (timedelta (date "2020-1-1") #:weeks 1 #:hours 2)
Produces a timezone-aware timestamp as of today
The tz
keyword allows to specify an alternate time zone.
The naive
keyword forces production of a naive timestamp.
Both tz
and naive
keywords are mutually exlcusive.
Example: (today)
A few api calls are added to the tshistory
base:
.register_formula
to define a formula
.eval_formula
to evaluate on-the-fly a formula (useful to check
that it computes before registering it)
Exemple:
tsa.register_formula(
'my-sweet-formula',
'(* 3.14 (series "going-round"))'
)
Example:
>>> tsa.eval_formula('(* 3.14 (series "going-round"))')
...
2020-01-01 3.14
2020-01-02 6.28
2020-01-03 9.42
dtype: float64