Shepherd-Core - Config-Models

Note

TODO: WORK IN PROGRESS

The models offer

one unified interface for all tools
auto-completion with neutral / sensible values
complex and custom datatypes (i.e. PositiveInt, lists-checks on length)
checking of inputs (validation) and type-casting
generate their own schema (for web-forms)
recursive inheritance (for content-configs)
pre-validation
store to & load from yaml or json with typecheck through wrapper
included documentation

Experiment

This category includes configuration-models for setting up an experiment. Part of the sub-systems are in the next section Observer Capabilities.

pydantic model shepherd_core.data_models.experiment.Experiment

Config for experiments on the testbed emulating energy environments for target nodes.

Fields:

name (str)
description (str | None)
comment (str | None)
email_results (bool)
sys_logging (shepherd_core.data_models.experiment.observer_features.SystemLogging)
time_start (datetime.datetime | None)
duration (datetime.timedelta | None)
target_configs (list[shepherd_core.data_models.experiment.target_config.TargetConfig])
lib_ver (str | None)

field name: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=32, pattern=^[^<>:;,?\"\*|\/\\]+$)] [Required]

Constraints:

max_length = 32
pattern = ^[^<>:;,?"*|/\]+$

field description: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern=^[ -~]+$)] | None, FieldInfo(annotation=NoneType, required=True, description='Required for public instances')] = None: Required for public instances

field comment: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern=^[ -~]+$)] | None = None

field email_results: bool = True

field sys_logging: SystemLogging = SystemLogging({})

field time_start: datetime | None = None

field duration: timedelta | None = None

field target_configs: Annotated[list[TargetConfig], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] [Required]

Constraints:

min_length = 1
max_length = 128

field lib_ver: str | None = '2025.06.4'

get_target_ids() → list

get_target_config(target_id: int) → TargetConfig

folder_name(custom_date: datetime | None = None) → str

pydantic model shepherd_core.data_models.experiment.TargetConfig

Configuration related to Target Nodes (DuT).

Fields:

target_IDs (list[int])
custom_IDs (list[int] | None)
energy_env (shepherd_core.data_models.content.energy_environment.EnergyEnvironment)
virtual_source (shepherd_core.data_models.content.virtual_source.VirtualSourceConfig)
target_delays (list[int] | None)
firmware1 (shepherd_core.data_models.content.firmware.Firmware)
firmware2 (shepherd_core.data_models.content.firmware.Firmware | None)
power_tracing (shepherd_core.data_models.experiment.observer_features.PowerTracing | None)
gpio_tracing (shepherd_core.data_models.experiment.observer_features.GpioTracing | None)
gpio_actuation (shepherd_core.data_models.experiment.observer_features.GpioActuation | None)
uart_logging (shepherd_core.data_models.experiment.observer_features.UartLogging | None)

field target_IDs: Annotated[list[Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=340282366920938463463374607431768211456)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] [Required]

Constraints:

min_length = 1
max_length = 128

field custom_IDs: Annotated[list[Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=65536)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] | None = None

⤷ custom ID will replace ‘const uint16_t SHEPHERD_NODE_ID’ in firmware.

if no custom ID is provided, the original ID of target is used

field energy_env: EnergyEnvironment [Required]: input for the virtual source

field virtual_source: VirtualSourceConfig = VirtualSourceConfig({'id': 1000, 'name': 'neutral', 'description': 'Direct feed-through of energy environment with no converter (allows on-off-patters)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True})

field target_delays: Annotated[list[Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] | None = None

⤷ individual starting times

allows to use the same environment
not implemented ATM

field firmware1: Firmware [Required]: ⤷ omitted FW gets set to neutral deep-sleep

field firmware2: Firmware | None = None: ⤷ omitted FW gets set to neutral deep-sleep

field power_tracing: PowerTracing | None = None

field gpio_tracing: GpioTracing | None = None

field gpio_actuation: GpioActuation | None = None

field uart_logging: UartLogging | None = None

get_custom_id(target_id: int) → int | None

Observer Capabilities

These are some of the sub-systems for configuring experiments and also tasks.

Link to Source

pydantic model shepherd_core.data_models.experiment.PowerTracing

Configuration for recording the Power-Consumption of the Target Nodes.

Fields:

intermediate_voltage (bool)
delay (datetime.timedelta)
duration (datetime.timedelta | None)
only_power (bool)
samplerate (int)

field intermediate_voltage: bool = False

⤷ for EMU: record storage capacitor instead of output (good for V_out = const): this also includes current!

field delay: timedelta = datetime.timedelta(0): start recording after experiment started

field duration: timedelta | None = None

duration of recording after delay starts the process.

default is None, recording till EOF

field only_power: bool = False: ⤷ reduce file-size by calculating power and automatically discard I&V

field samplerate: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=10), Le(le=100000)])] = 100000

⤷ reduce file-size by re-sampling (mean over x samples) Timestamps will be taken from the start of that sample-window. example: 10 Hz samplerate will be binning 10k samples via mean() and

the timestamp for that new sample will be value[0] of the 10k long vector

Constraints:

ge = 10
le = 100000

pydantic model shepherd_core.data_models.experiment.GpioTracing

Configuration for recording the GPIO-Output of the Target Nodes.

TODO: postprocessing not implemented ATM

Fields:

gpios (list[int])
delay (datetime.timedelta)
duration (datetime.timedelta | None)
uart_decode (bool)
uart_pin (shepherd_core.data_models.testbed.gpio.GPIO)
uart_baudrate (int)

field gpios: Annotated[list[Annotated[int, Interval(gt=None, ge=0, lt=None, le=17)]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=18)])] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]

List of GPIO to record.

This feature allows to remove unwanted pins from recording, i.e. for chatty pins with separate UART Logging enabled. Numbering is based on the Target-Port and its 16x GPIO and two PwrGood-Signals. See doc for nRF_FRAM_Target_v1.3+ to see mapping of target port.

Example for skipping UART (pin 0 & 1): .gpio = range(2,18)

Note: - Cape 2.4 (2023) and lower only has 9x GPIO + 1x PwrGood - Cape 2.5 (2025) has first 12 GPIO & both PwrGood - this will be mapped accordingly by the observer

Constraints:

min_length = 1
max_length = 18

field delay: timedelta = datetime.timedelta(0)

field duration: timedelta | None = None

field uart_decode: bool = False: Automatic decoding from gpio-trace not implemented ATM.

field uart_pin: GPIO = GPIO({'id': 1008, 'name': 'GPIO8', 'description': 'alias UART_TX', 'direction': 'IO', 'dir_switch': 'DIR2', 'reg_pru': 'r31_08', 'pin_pru': 'P8_27', 'reg_sys': 15, 'pin_sys': 'P9_24'})

field uart_baudrate: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=2400), Le(le=1152000)])] = 115200

Constraints:

ge = 2400
le = 1152000

property gpio_mask: int

deactiveated due to Error (TODO)
.. autopydantic_model:: shepherd_core.data_models.experiment.GpioActuation
.. autopydantic_model:: shepherd_core.data_models.experiment.GpioLevel

pydantic model shepherd_core.data_models.experiment.SystemLogging

Configuration for recording Debug-Output of the Observers System-Services.

Fields:

kernel (bool)
time_sync (bool)
sheep (bool)
sys_util (bool)

field kernel: bool = True

field time_sync: bool = True

field sheep: bool = True

field sys_util: bool = True

Content-Types

Reusable user-defined meta-data for fw, h5 and vsrc-definitions.

Link to Source

pydantic model shepherd_core.data_models.content.EnergyEnvironment

Recording of meta-data representation of a testbed-component.

Fields:

data_path (pathlib.Path)
data_type (shepherd_core.data_models.content.energy_environment.EnergyDType)
data_local (bool)
duration (float)
energy_Ws (float)
valid (bool)
light_source (str | None)
weather_conditions (str | None)
indoor (bool | None)
location (str | None)

field data_path: Path [Required]

field data_type: EnergyDType [Required]

field data_local: bool = True: ⤷ signals that file has to be copied to testbed

field duration: Annotated[float, Gt(gt=0)] [Required]

Constraints:

gt = 0

field energy_Ws: Annotated[float, Gt(gt=0)] [Required]

Constraints:

gt = 0

field valid: bool = False

field light_source: str | None = None

field weather_conditions: str | None = None

field indoor: bool | None = None

field location: str | None = None

pydantic model shepherd_core.data_models.content.Firmware

meta-data representation of a data-component.

Fields:

mcu (shepherd_core.data_models.testbed.mcu.MCU)
data (str | pathlib.Path)
data_type (shepherd_core.data_models.content.firmware_datatype.FirmwareDType)
data_hash (str | None)
data_local (bool)

field mcu: MCU [Required]

field data: Annotated[str, StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=3, max_length=8000000, pattern=None)] | Path [Required]

field data_type: FirmwareDType [Required]

field data_hash: str | None = None

field data_local: bool = True: ⤷ signals that file has to be copied to testbed

classmethod from_firmware(file: Path, *, embed: bool = True, **kwargs: Unpack[TypedDict]) → Self

Embeds firmware and tries to fill parameters.

ELF -> mcu und data_type are deducted HEX -> must supply mcu manually.

compare_hash(path: Path | None = None) → bool

extract_firmware(file: Path) → Path

Store embedded fw-data in file.

file-suffix is derived from data-type and adapted
if provided path is a directory, the firmware-name is used

pydantic model shepherd_core.data_models.content.VirtualHarvesterConfig

The virtual harvester configuration characterizes usage of an energy environment.

It is used to both harvesting during emulation and to record energy environments (sometimes referred to as “harvesting traces”).

For emulation:

The virtual harvester configuration describes how energy from a recorded energy environment is harvested. Typically, the energy environment provides an IV-surface, which is a continuous function in three dimensions: voltage, current, and time. Based on this surface, the emulation can derive the available IV-curve at each point in time. The harvester looks up the current that is available (according to the energy environment) from a given harvesting voltage. The harvesting voltage may be dynamically chosen by the harvester based on the implemented harvesting algorithm, which models different real-world harvesters. For example, a maximum power point tracking harvester may choose a harvesting voltage as a ratio of the open circuit voltage available from the energy environment (or transducer in practice).

The energy environments are encoded not as a three-dimensional function, but as IV tuples over time (sampled at a constant frequency). This originates from the technical implementation when recording the IV-surface, where the recorder provides the IV-curve by measuring the current for a given voltage and ramping the voltage from minimal to maximum.

For harvest-recordings:

An energy environment is fully described by the IV surface, which are IV curves over time. Shepherd approximates this by sampling the current at equidistant steps of a voltage ramp. The VirtualHarvesterConfig is also used to parameterize the recording process, typically, it should be configured to record a full IV surface, as this contains the full information of the energy environment. The postponed harvesting is then performed during emulation.

However, it is also possible to record a “pre-harvested” energy environment by performing the harvesting during recording. This results in a recording containing IV samples over time that represent the harvesting voltage (chosen by the virtual harvester during recording) and the current available from the energy environment for that voltage. Together, these represent the power available for harvesting at the time, and during emulation, this power can be converted by the input stage (boost converter) to charge the energy storage.

Fields:

algorithm (shepherd_core.data_models.content.virtual_harvester.AlgorithmDType)
samples_n (int)
voltage_mV (float)
voltage_min_mV (float)
voltage_max_mV (float)
current_limit_uA (float)
voltage_step_mV (float | None)
setpoint_n (float)
interval_ms (float)
duration_ms (float)
rising (bool)
enable_linear_extrapolation (bool)
wait_cycles (int)

field algorithm: AlgorithmDType [Required]: The algorithm determines how the harvester chooses the harvesting voltage.

field samples_n: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=8), Le(le=2000)])] = 8

How many IV samples are measured for one IV curve.

The curve is recorded by measuring the el. current available from the transducer at equidistant voltage intervals. These voltages are probed by ramping between voltage_min_mV and voltage_max_mV at samples_n points equally distributed over the voltage range. After setting the voltage, the recorder waits for a short period - allowing the analog frontend and transducer to settle - before recording the harvesting current. This wait duration is influenced by wait_cycles.

Selecting all these parameters is a tradeoff between accuracy of the IV curve (density of IV samples) and measurement duration, hence the time accuracy (density of points) of the IV-surface.

Only applicable to recording, not used in emulation.

Used together with voltage_min_mV, voltage_max_mV, rising, and wait_cycles.

Constraints:

ge = 8
le = 2000

field voltage_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 2500

The harvesting voltage for constant voltage harvesting.

Additionally, for Perturb-and-Observe MPPT, this defines the voltage at startup.

Constraints:

ge = 0
le = 5000

field voltage_min_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 0

Minimum voltage recorded for the IV curve.

See samples_n for further details.

In emulation, this can be used to “block” parts of the recorded IV curve and not utilize them in the virtual source. However, this is generally discouraged as it can result in discontinuities in the curve and is not well tested. For emulation, this value ideally corresponds to the value of the recorded energy environment.

Constraints:

ge = 0
le = 5000

field voltage_max_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 5000

Maximum voltage sampled for the curve.

See voltage_min_mV and samples_n.

Constraints:

ge = 0
le = 5000

field current_limit_uA: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=50000)])] = 50000

For MPPT VOC, the open circuit voltage is identified as the el. current crosses below this threshold.

During recording it allows to keep trajectory in special region (constant current tracking).

Constraints:

ge = 1
le = 50000

field voltage_step_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=1000000)])] | None = None

The difference between two adjacent voltage samples.

This value is implicitly derived from the other ramp parameters: (voltage_max_mV - voltage_min_mV) / (samples_n - 1)

field setpoint_n: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=1.0)])] = 0.7

The “Open Circuit Voltage Maximum Power Point Tracker” estimates the MPP by taking a constant fraction defined by this parameter of the open circuit voltage. For example, if the IV curve shows an open circuit voltage of 2V and the setpoint is 0.75, then the harvester selects 1.5 volts as the harvesting voltage.

This value is only relevant when ‘algorithm == mppt_voc’.

Constraints:

ge = 0
le = 1.0

field interval_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.01), Le(le=1000000)])] = 100

The MPP is repeatedly estimated at fixed intervals defined by this duration.

Note that the energy environment can still change in between MPP estimations, but the harvesting voltage is not updated in between.

This value is relevant for all MPP algorithms. For Perturb and Observe, this value is the wait interval between steps.

When an energy environment is recorded with mppt_opt, the optimal harvester is approximated with a very fast Perturb-Observe algorithm, where this interval should be set to a very small value. When emulating with mppt_opt, this value is not relevant as the emulation simply picks the maximum power point from the IV-curve.

Constraints:

ge = 0.01
le = 1000000

field duration_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.01), Le(le=1000000)])] = 0.1

The duration of MPP sampling.

While performing an MPP sampling every ‘interval_ms’, the input is disconnected to accurately measure the open circuit voltage.

This value is only relevant for mppt_voc.

Constraints:

ge = 0.01
le = 1000000

field rising: bool = True

Ramp direction for sampling the IV curve.

When set to true, sampling starts at the minimum voltage and ramps up to the maximum.

See samples_n for further details. Not relevant for emulation.

field enable_linear_extrapolation: bool = True

Because the IV curve is not stored fully in PRU memory but streamed sequentially to the PRU, looking up any IV value at any time is not possible. However, because the precision of the emulation degrades until the relevant IV sample passes by, it can extrapolate the available data to estimate the required IV sample.

Enabling extrapolation can yield a better numeric simulation, especially if the harvesting voltage changes rapidly or the IV surface is steep in relevant regions. For example, when emulating a capacitor diode setup and the current falls at high voltages.

This value is only relevant for emulation.

field wait_cycles: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=100)])] = 1

The wait duration to let the analog frontend settle before taking a measurement.

When recording the energy environment, the voltage is set by the digital-to-analog-converter. This parameter delays the current measurement performed by the analog-to-digital converter to allow the harvesting transducer to settle at the defined voltage.

When recording with IscVoc, wait cycles should be added as the analog changes are more significant.

Not relevant for emulation.

Constraints:

ge = 0
le = 100

calc_hrv_mode(*, for_emu: bool) → int

calc_algorithm_num(*, for_emu: bool) → int

calc_timings_ms(*, for_emu: bool) → tuple[float, float]: factor-in model-internal timing-constraints.

get_datatype() → EnergyDType

calc_window_size(dtype_in: EnergyDType | None = None, *, for_emu: bool) → int

pydantic model shepherd_core.data_models.content.VirtualSourceConfig

The vSrc uses the energy environment (file) for supplying the Target Node.

If not already done, the energy will be harvested and then converted during the experiment.

The converter-stage is software defined and offers: - buck-boost-combinations, - a simple diode + resistor and - an intermediate storage capacitor.

Fields:

enable_boost (bool)
enable_buck (bool)
enable_feedback_to_hrv (bool)
interval_startup_delay_drain_ms (float)
harvester (shepherd_core.data_models.content.virtual_harvester.VirtualHarvesterConfig)
V_input_max_mV (float)
I_input_max_mA (float)
V_input_drop_mV (float)
R_input_mOhm (float)
C_intermediate_uF (float)
V_intermediate_init_mV (float)
I_intermediate_leak_nA (float)
V_intermediate_enable_threshold_mV (float)
V_intermediate_disable_threshold_mV (float)
interval_check_thresholds_ms (float)
V_pwr_good_enable_threshold_mV (float)
V_pwr_good_disable_threshold_mV (float)
immediate_pwr_good_signal (bool)
C_output_uF (float)
V_output_log_gpio_threshold_mV (float)
V_input_boost_threshold_mV (float)
V_intermediate_max_mV (float)
LUT_input_efficiency (list[list[float]])
LUT_input_V_min_log2_uV (int)
LUT_input_I_min_log2_nA (int)
V_output_mV (float)
V_buck_drop_mV (float)
LUT_output_efficiency (list[float])
LUT_output_I_min_log2_nA (int)

field enable_boost: bool = False: ⤷ if false -> v_intermediate = v_input, output-switch-hysteresis is still usable

field enable_buck: bool = False: ⤷ if false -> v_output = v_intermediate

field enable_feedback_to_hrv: bool = False: src can control a cv-harvester for ivcurve

field interval_startup_delay_drain_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 0

Constraints:

ge = 0
le = 10000

field harvester: VirtualHarvesterConfig = VirtualHarvesterConfig({'id': 2206, 'name': 'mppt_opt', 'description': 'Power-Optimum with very fast PO-Variant (harvesting) or special max-pwr-picker (emulator / ivcurve)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True, 'algorithm': 'mppt_opt', 'voltage_step_mV': 1.0, 'interval_ms': 0.01})

field V_input_max_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 10000

Constraints:

ge = 0
le = 10000

field I_input_max_mA: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290.0)])] = 4200

Constraints:

ge = 0
le = 4290.0

field V_input_drop_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 0

⤷ simulate input-diode

Constraints:

ge = 0
le = 4290000.0

field R_input_mOhm: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 0

⤷ resistance only active with disabled boost, range [1 mOhm; 1MOhm]

Constraints:

ge = 0
le = 4290000.0

field C_intermediate_uF: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=100000)])] = 0

Constraints:

ge = 0
le = 100000

field V_intermediate_init_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 3000

⤷ allow a proper / fast startup

Constraints:

ge = 0
le = 10000

field I_intermediate_leak_nA: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000000.0)])] = 0

Constraints:

ge = 0
le = 4290000000.0

field V_intermediate_enable_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 1

⤷ target gets connected (hysteresis-combo with next value)

Constraints:

ge = 0
le = 10000

field V_intermediate_disable_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 0

⤷ target gets disconnected

Constraints:

ge = 0
le = 10000

field interval_check_thresholds_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290.0)])] = 0

⤷ some ICs (BQ) check every 64 ms if output should be disconnected

Constraints:

ge = 0
le = 4290.0

field V_pwr_good_enable_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 2800

Constraints:

ge = 0
le = 10000

field V_pwr_good_disable_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 2200

Constraints:

ge = 0
le = 10000

field immediate_pwr_good_signal: bool = True: ⤷ 1: activate instant schmitt-trigger, 0: stay in interval for checking thresholds

field C_output_uF: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 1.0

final (always last) stage to compensate undetectable current spikes when enabling power for target

Constraints:

ge = 0
le = 4290000.0

field V_output_log_gpio_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 1400

⤷ min voltage needed to enable recording changes in gpio-bank

Constraints:

ge = 0
le = 4290000.0

field V_input_boost_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 0

⤷ min input-voltage for the boost converter to work

Constraints:

ge = 0
le = 10000

field V_intermediate_max_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 10000

⤷ boost converter shuts off

Constraints:

ge = 0
le = 10000

field LUT_input_efficiency: Annotated[list[Annotated[list[Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.0), Le(le=1.0)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=12), MaxLen(max_length=12)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=12), MaxLen(max_length=12)])] = [[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]]

⤷ rows are current -> first row a[V=0][:]

input-LUT[12][12] depending on array[inp_voltage][log(inp_current)], influence of cap-voltage is not implemented

Constraints:

min_length = 12
max_length = 12

field LUT_input_V_min_log2_uV: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=20)])] = 0

⤷ i.e. 2^7 = 128 uV -> LUT[0][:] is for inputs < 128 uV

Constraints:

ge = 0
le = 20

field LUT_input_I_min_log2_nA: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=20)])] = 1

⤷ i.e. 2^8 = 256 nA -> LUT[:][0] is for inputs < 256 nA

Constraints:

ge = 1
le = 20

field V_output_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 2400

Constraints:

ge = 0
le = 5000

field V_buck_drop_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 0

⤷ simulate LDO / diode min voltage differential or output-diode

Constraints:

ge = 0
le = 5000

field LUT_output_efficiency: Annotated[list[Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.0), Le(le=1.0)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=12), MaxLen(max_length=12)])] = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

⤷ array[12] depending on output_current

Constraints:

min_length = 12
max_length = 12

field LUT_output_I_min_log2_nA: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=20)])] = 1

⤷ 2^8 = 256 nA -> LUT[0] is for inputs < 256 nA, see notes on LUT_input for explanation

Constraints:

ge = 1
le = 20

calc_internal_states() → dict

Update the model-states for the capacitor and other elements.

This also compensates for current-surge of real capacitors when the converter gets turned on:

surges are hard to detect & record
this can be const value, because
the converter always turns on with “V_storage_enable_threshold_uV”.

TODO: currently neglecting delay after disabling converter, boost only has simpler formula, second enabling when V_Cap >= V_out

Math behind this calculation:

Energy-Change Storage Cap -> E_new = E_old - E_output
with Energy of a Cap -> E_x = C_x * V_x^2 / 2
combine formulas -> C_store * V_store_new^2 / 2 = C_store * V_store_old^2 / 2 - C_out * V_out^2 / 2
convert formula to V_new -> V_store_new^2 = V_store_old^2 - (C_out / C_store) * V_out^2
convert into dV -> dV = V_store_new - V_store_old
in case of V_cap = V_out -> dV = V_store_old * (sqrt(1 - C_out / C_store) - 1)

Note: dV values will be reversed (negated), because dV is always negative (Voltage drop)

calc_converter_mode(dtype_in: EnergyDType, *, log_intermediate_node: bool) → int

Assembles bitmask from discrete values.

log_intermediate_node: record / log virtual intermediate (cap-)voltage and -current (out) instead of output-voltage and -current

calc_cap_constant_us_per_nF_n28() → int

Calc constant to convert capacitor-current to Voltage-delta.

dV[uV] = constant[us/nF] * current[nA] = constant[us*V/nAs] * current[nA]

Tasks

These are digestible configs for shepherd-herd or -sheep.

Link to Source

pydantic model shepherd_core.data_models.task.HarvestTask

Config for the Observer in Harvest-Mode to record IV data from a harvesting-source.

Fields:

output_path (pathlib.Path)
force_overwrite (bool)
output_compression (shepherd_core.data_models.task.emulation.Compression | None)
time_start (datetime.datetime | None)
duration (datetime.timedelta | None)
use_cal_default (bool)
virtual_harvester (shepherd_core.data_models.content.virtual_harvester.VirtualHarvesterConfig)
power_tracing (shepherd_core.data_models.experiment.observer_features.PowerTracing)
sys_logging (shepherd_core.data_models.experiment.observer_features.SystemLogging | None)
verbose (int)

field output_path: Path [Required]

⤷ dir- or file-path for storing the recorded data:

providing a directory -> file is named hrv_timestamp.h5
for a complete path the filename is not changed except it exists and overwrite is disabled -> name#num.h5

field force_overwrite: bool = False: ⤷ Overwrite existing file

field output_compression: Compression | None = Compression.lzf: ⤷ should be 1 (level 1 gzip), lzf, or None (order of recommendation)

field time_start: datetime | None = None: timestamp or unix epoch time, None = ASAP

field duration: timedelta | None = None: ⤷ Duration of recording in seconds, None = till EOFSys

field use_cal_default: bool = False: ⤷ Use default calibration values, skip loading from EEPROM

field virtual_harvester: VirtualHarvesterConfig = VirtualHarvesterConfig({'id': 2206, 'name': 'mppt_opt', 'description': 'Power-Optimum with very fast PO-Variant (harvesting) or special max-pwr-picker (emulator / ivcurve)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True, 'algorithm': 'mppt_opt', 'voltage_step_mV': 1.0, 'interval_ms': 0.01}): ⤷ Choose one of the predefined virtual harvesters or configure a new one

field power_tracing: PowerTracing = PowerTracing({})

field sys_logging: SystemLogging | None = SystemLogging({})

field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug

Constraints:

ge = 0
le = 4

is_contained(paths: Set[PurePosixPath]) → bool

pydantic model shepherd_core.data_models.task.EmulationTask

Configuration for the Observer in Emulation-Mode.

Fields:

input_path (pathlib.Path)
output_path (pathlib.Path | None)
force_overwrite (bool)
output_compression (shepherd_core.data_models.task.emulation.Compression | None)
time_start (datetime.datetime | None)
duration (datetime.timedelta | None)
use_cal_default (bool)
enable_io (bool)
io_port (shepherd_core.data_models.testbed.cape.TargetPort)
pwr_port (shepherd_core.data_models.testbed.cape.TargetPort)
voltage_aux (float | str)
virtual_source (shepherd_core.data_models.content.virtual_source.VirtualSourceConfig)
power_tracing (shepherd_core.data_models.experiment.observer_features.PowerTracing | None)
gpio_tracing (shepherd_core.data_models.experiment.observer_features.GpioTracing | None)
uart_logging (shepherd_core.data_models.experiment.observer_features.UartLogging | None)
gpio_actuation (shepherd_core.data_models.experiment.observer_features.GpioActuation | None)
sys_logging (shepherd_core.data_models.experiment.observer_features.SystemLogging | None)
verbose (int)

field input_path: Path [Required]: ⤷ hdf5 file containing harvesting data

field output_path: Path | None = None

⤷ dir- or file-path for storing the recorded data:

providing a directory -> file is named emu_timestamp.h5
for a complete path the filename is not changed except it exists and overwrite is disabled -> emu#num.h5

TODO: should the output-path be mandatory?

field force_overwrite: bool = False: ⤷ Overwrite existing file

field output_compression: Compression | None = Compression.lzf: ⤷ should be lzf, 1 (gzip level 1) or None (order of recommendation)

field time_start: datetime | None = None: timestamp or unix epoch time, None = ASAP

field duration: timedelta | None = None: ⤷ Duration of recording in seconds, None = till EOF

field use_cal_default: bool = False: ⤷ Use default calibration values, skip loading from EEPROM

field enable_io: bool = True

⤷ Switch the GPIO level converter to targets on/off

pre-req for sampling gpio / uart,

field io_port: TargetPort = TargetPort.A: ⤷ Either Port A or B that gets connected to IO

field pwr_port: TargetPort = TargetPort.A

⤷ selected port will be current-monitored

main channel is nnected to virtual Source
the other port is aux

field voltage_aux: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4.5)])] | str = 0: ⤷ aux_voltage options - 0-4.5 for specific const Voltage (0 V = disabled), - “buffer” will output intermediate voltage (storage cap of vsource), - “main” will mirror main target voltage

field virtual_source: VirtualSourceConfig = VirtualSourceConfig({'id': 1000, 'name': 'neutral', 'description': 'Direct feed-through of energy environment with no converter (allows on-off-patters)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True})

⤷ Use the desired setting for the virtual source,

provide parameters or name like BQ25570

field power_tracing: PowerTracing | None = PowerTracing({})

field gpio_tracing: GpioTracing | None = GpioTracing({})

field uart_logging: UartLogging | None = UartLogging({})

field gpio_actuation: GpioActuation | None = None

field sys_logging: SystemLogging | None = SystemLogging({})

field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug,

TODO: just bool now, systemwide

Constraints:

ge = 0
le = 4

classmethod from_xp(xp: Experiment, tb: Testbed, tgt_id: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=340282366920938463463374607431768211456)])], root_path: Path) → Self

is_contained(paths: Set[PurePosixPath]) → bool

pydantic model shepherd_core.data_models.task.FirmwareModTask

Config for Task that adds the custom ID to the firmware & stores it into a file.

Fields:

data (str | pathlib.Path)
data_type (shepherd_core.data_models.content.firmware_datatype.FirmwareDType)
custom_id (int | None)
firmware_file (pathlib.Path)
verbose (int)

field data: Annotated[str, StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=3, max_length=8000000, pattern=None)] | Path [Required]

field data_type: FirmwareDType [Required]

field custom_id: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=65536)])] | None = None

field firmware_file: Path [Required]

field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug

Constraints:

ge = 0
le = 4

classmethod from_xp(xp: Experiment, tb: Testbed, tgt_id: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=340282366920938463463374607431768211456)])], mcu_port: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=2)])], fw_path: Path) → Self

classmethod from_firmware(fw: Firmware, **kwargs: Unpack[TypedDict]) → Self

is_contained(paths: Set[PurePosixPath]) → bool

pydantic model shepherd_core.data_models.task.ProgrammingTask

Config for a Task programming the selected target.

Fields:

firmware_file (pathlib.Path)
target_port (shepherd_core.data_models.testbed.cape.TargetPort)
mcu_port (int)
mcu_type (str)
voltage (float)
datarate (int)
protocol (shepherd_core.data_models.testbed.mcu.ProgrammerProtocol)
simulate (bool)
verbose (int)

field firmware_file: Path [Required]

field target_port: TargetPort = TargetPort.A

field mcu_port: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=2)])] = 1

Constraints:

ge = 1
le = 2

field mcu_type: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern=^[ -~]+$)] [Required]

⤷ must be either “nrf52” or “msp430” ATM, TODO: clean xp to tasks

Constraints:

pattern = ^[ -~]+$

field voltage: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Lt(lt=5)])] = 3

Constraints:

ge = 1
lt = 5

field datarate: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Gt(gt=0), Le(le=1000000)])] = 200000

Constraints:

gt = 0
le = 1000000

field protocol: ProgrammerProtocol [Required]

field simulate: bool = False

field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug

Constraints:

ge = 0
le = 4

classmethod from_xp(xp: Experiment, tb: Testbed, tgt_id: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=340282366920938463463374607431768211456)])], mcu_port: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=2)])], fw_path: Path) → Self | None

is_contained(paths: Set[PurePosixPath]) → bool

pydantic model shepherd_core.data_models.task.ObserverTasks

Collection of tasks for selected observer included in experiment.

Fields:

observer (str)
time_prep (datetime.datetime | None)
root_path (pathlib.Path)
fw1_mod (shepherd_core.data_models.task.firmware_mod.FirmwareModTask | None)
fw2_mod (shepherd_core.data_models.task.firmware_mod.FirmwareModTask | None)
fw1_prog (shepherd_core.data_models.task.programming.ProgrammingTask | None)
fw2_prog (shepherd_core.data_models.task.programming.ProgrammingTask | None)
emulation (shepherd_core.data_models.task.emulation.EmulationTask | None)
owner_id (int | None)
abort_on_error (bool)

field observer: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=32, pattern=^[^<>:;,?\"\*|\/\\]+$)] [Required]

Constraints:

max_length = 32
pattern = ^[^<>:;,?"*|/\]+$

field time_prep: datetime | None = None

field root_path: Path [Required]

field fw1_mod: FirmwareModTask | None = None

field fw2_mod: FirmwareModTask | None = None

field fw1_prog: ProgrammingTask | None = None

field fw2_prog: ProgrammingTask | None = None

field emulation: EmulationTask | None = None

owner_id: deprecated object at 0x7fe09a1a97f0>]

Read-only data descriptor used to emit a runtime deprecation warning before accessing a deprecated field.

msg: The deprecation message to be emitted.

wrapped_property: The property instance if the deprecated field is a computed field, or None.

field_name: The name of the field being deprecated.

abort_on_error: deprecated object at 0x7fe09a1aa870>]

Read-only data descriptor used to emit a runtime deprecation warning before accessing a deprecated field.

msg: The deprecation message to be emitted.

wrapped_property: The property instance if the deprecated field is a computed field, or None.

field_name: The name of the field being deprecated.

classmethod from_xp(xp: Experiment, tb: Testbed, tgt_id: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=340282366920938463463374607431768211456)])]) → Self

get_tasks() → list[ShpModel]

get_output_paths() → dict[str, Path]

is_contained(paths: Set[PurePosixPath]) → bool

pydantic model shepherd_core.data_models.task.TestbedTasks

Collection of tasks for all observers included in experiment.

Fields:

name (str)
observer_tasks (list[shepherd_core.data_models.task.observer_tasks.ObserverTasks])

field name: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=32, pattern=^[^<>:;,?\"\*|\/\\]+$)] [Required]

Constraints:

max_length = 32
pattern = ^[^<>:;,?"*|/\]+$

field observer_tasks: Annotated[list[ObserverTasks], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] [Required]

Constraints:

min_length = 1
max_length = 128

classmethod from_xp(xp: Experiment, tb: Testbed | None = None) → Self

get_observer_tasks(observer: str) → ObserverTasks | None

get_output_paths() → dict[str, Path]

is_contained() → bool