Shepherd-Core - Config-Models

Note

TODO: WORK IN PROGRESS

The models offer

  • one unified interface for all tools

  • auto-completion with neutral / sensible values

  • complex and custom datatypes (i.e. PositiveInt, lists-checks on length)

  • checking of inputs (validation) and type-casting

  • generate their own schema (for web-forms)

  • recursive inheritance (for content-configs)

  • pre-validation

  • store to & load from yaml or json with typecheck through wrapper

  • included documentation

Experiment

This category includes configuration-models for setting up an experiment. Part of the sub-systems are in the next section Observer Capabilities.

pydantic model shepherd_core.data_models.experiment.Experiment

Config for experiments on the testbed emulating energy environments for target nodes.

Fields:

  • name (str)

  • description (str | None)

  • comment (str | None)

  • email_results (bool)

  • sys_logging (shepherd_core.data_models.experiment.observer_features.SystemLogging)

  • time_start (datetime.datetime | None)

  • duration (datetime.timedelta | None)

  • target_configs (list[shepherd_core.data_models.experiment.target_config.TargetConfig])

  • lib_ver (str | None)

field name: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=50, pattern=^[^<>:;,?\"\*|\/\\]+$)] [Required]
Constraints:

  • max_length = 50

  • pattern = ^[^<>:;,?"*|/\]+$

field description: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern=^[ -~]+$)] | None, FieldInfo(annotation=NoneType, required=True, description='Required for public instances')] = None

Required for public instances

field comment: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern=^[ -~]+$)] | None = None
field email_results: bool = True
field sys_logging: SystemLogging = SystemLogging({})
field time_start: datetime | None = None
field duration: timedelta | None = None
field target_configs: Annotated[list[TargetConfig], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] [Required]
Constraints:

  • min_length = 1

  • max_length = 128

field lib_ver: str | None = '2026.02.4'
pydantic model shepherd_core.data_models.experiment.TargetConfig

Configuration related to Target Nodes (DuT).

Fields:

  • target_IDs (list[int])

  • custom_IDs (list[int] | None)

  • energy_env (shepherd_core.data_models.content.energy_environment.EnergyEnvironment)

  • virtual_source (shepherd_core.data_models.content.virtual_source_config.VirtualSourceConfig)

  • target_delays (list[int] | None)

  • firmware1 (shepherd_core.data_models.content.firmware.Firmware)

  • firmware2 (shepherd_core.data_models.content.firmware.Firmware | None)

  • power_tracing (shepherd_core.data_models.experiment.observer_features.PowerTracing | None)

  • gpio_tracing (shepherd_core.data_models.experiment.observer_features.GpioTracing | None)

  • gpio_actuation (shepherd_core.data_models.experiment.observer_features.GpioActuation | None)

  • uart_logging (shepherd_core.data_models.experiment.observer_features.UartLogging | None)

field target_IDs: Annotated[list[Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=340282366920938463463374607431768211456)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] [Required]
Constraints:

  • min_length = 1

  • max_length = 128

field custom_IDs: Annotated[list[Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=65536)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] | None = None

⤷ custom ID will replace ‘const uint16_t SHEPHERD_NODE_ID’ in firmware.

if no custom ID is provided, the original ID of target is used

field energy_env: EnergyEnvironment [Required]

input for the virtual source

field virtual_source: VirtualSourceConfig = VirtualSourceConfig({'id': 1000, 'name': 'neutral', 'description': 'Direct feed-through of energy environment with no converter (allows on-off-patters)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True})
field target_delays: Annotated[list[Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] | None = None

⤷ individual starting times

  • allows to use the same environment

  • not implemented ATM

field firmware1: Firmware [Required]

⤷ omitted FW gets set to neutral deep-sleep

field firmware2: Firmware | None = None

⤷ omitted FW gets set to neutral deep-sleep

field power_tracing: PowerTracing | None = None
field gpio_tracing: GpioTracing | None = None
field gpio_actuation: GpioActuation | None = None
field uart_logging: UartLogging | None = None

Observer Capabilities

These are some of the sub-systems for configuring experiments and also tasks.

Link to Source

pydantic model shepherd_core.data_models.experiment.PowerTracing

Configuration for recording the Power-Consumption of the Target Nodes.

With the default configuration voltage and current are sampled with 100 kHz.

Fields:

  • intermediate_voltage (bool)

  • delay (datetime.timedelta)

  • duration (datetime.timedelta | None)

  • only_power (bool)

  • samplerate (int)

field intermediate_voltage: bool = False
⤷ for EMU: record output-path of intermediate energy storage (capacitor, battery)

instead of direct target voltage-output (good for V_out = const) this also includes current!

field delay: timedelta = datetime.timedelta(0)

start recording after experiment started

field duration: timedelta | None = None

duration of recording after delay starts the process.

default is None, recording till EOF

field only_power: bool = False

⤷ reduce file-size by calculating power and automatically discard I&V Caution: increases cpu-utilization on observer

sampling power @ 100 kHz is not recommended

field samplerate: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=10), Le(le=100000)])] = 100000

⤷ reduce file-size by re-sampling (mean over x samples)

Caution: this option is available for IV-Samples (not only .only_power)

but results might be faulty for use-cases that are not const-voltage Sum(U * I) != Sum(U) * Sum(I)

Timestamps will be taken from the start of that sample-window. example: 10 Hz samplerate will be binning 10k samples via mean() and

the timestamp for that new sample will be value[0] of the 10k long vector

Constraints:

  • ge = 10

  • le = 100000

pydantic model shepherd_core.data_models.experiment.GpioTracing

Configuration for recording the GPIO-Output of the Target Nodes.

TODO: postprocessing not implemented ATM

Fields:

  • gpios (list[int])

  • delay (datetime.timedelta)

  • duration (datetime.timedelta | None)

  • uart_decode (bool)

  • uart_pin (shepherd_core.data_models.testbed.gpio.GPIO)

  • uart_baudrate (int)

field gpios: Annotated[list[Annotated[int, Interval(gt=None, ge=0, lt=None, le=17)]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=18)])] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]

List of GPIO to record.

This feature allows to remove unwanted pins from recording, i.e. for chatty pins with separate UART Logging enabled. Numbering is based on the Target-Port and its 16x GPIO and two PwrGood-Signals. See doc for nRF_FRAM_Target_v1.3+ to see mapping of target port.

Example for skipping UART (pin 0 & 1): .gpios = range(2,18)

Note: - Cape 2.4 (2023) and lower only has 9x GPIO + 1x PwrGood - Cape 2.5 (2025) has first 12 GPIO & both PwrGood - this will be mapped accordingly by the observer

Constraints:

  • min_length = 1

  • max_length = 18

field delay: timedelta = datetime.timedelta(0)
field duration: timedelta | None = None
field uart_decode: bool = False

Automatic decoding from gpio-trace not implemented ATM.

field uart_pin: GPIO = GPIO({'id': 1008, 'name': 'GPIO8', 'description': 'alias UART_TX', 'direction': 'IO', 'dir_switch': 'DIR2', 'reg_pru': 'r31_08', 'pin_pru': 'P8_27', 'reg_sys': 15, 'pin_sys': 'P9_24'})
field uart_baudrate: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=2400), Le(le=1152000)])] = 115200
Constraints:

  • ge = 2400

  • le = 1152000

deactiveated due to Error (TODO)
.. autopydantic_model:: shepherd_core.data_models.experiment.GpioActuation
.. autopydantic_model:: shepherd_core.data_models.experiment.GpioLevel
pydantic model shepherd_core.data_models.experiment.SystemLogging

Configuration for recording Debug-Output of the Observers System-Services.

Fields:

  • kernel (bool)

  • time_sync (bool)

  • sheep (bool)

  • sys_util (bool)

field kernel: bool = True
field time_sync: bool = True
field sheep: bool = True
field sys_util: bool = True

Content-Types

Reusable user-defined meta-data for fw, h5 and vsrc-definitions.

Link to Source

pydantic model shepherd_core.data_models.content.EnergyEnvironment

Metadata representation of spatio-temporal energy-recording.

Fields:

  • energy_profiles (list[shepherd_core.data_models.content.energy_environment.EnergyProfile])

  • metadata (collections.abc.Mapping[str, str | int | float])

  • modifications (collections.abc.Sequence[str])

  • PROFILES_MAX (int)

field energy_profiles: Annotated[list[EnergyProfile], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])] [Required]

⤷ list of individual profiles that make up the environment

Constraints:

  • min_length = 1

field metadata: Mapping[str, str | int | float] = {}

⤷ additional descriptive information

Example for solar: (main) light source, weather conditions, indoor General: transducer / harvester used, date, time, experiment setup, location, route

field modifications: Sequence[str] = []

Changes recorded by manipulation-Ops (i.e. addition, slicing).

field PROFILES_MAX: int = 128

⤷ arbitrary maximum, internal state which controls behavior for repetitions_ok-cases

  • single item list access is possible as modulo

  • sliced list access repeats profile-list up to max length

    ee[10:] gets (max - 10) items

pydantic model shepherd_core.data_models.content.Firmware

meta-data representation of a data-component.

Fields:

  • mcu (shepherd_core.data_models.testbed.mcu.MCU)

  • data (pathlib.Path | str)

  • data_type (shepherd_core.data_models.content.enum_datatypes.FirmwareDType)

  • data_hash (str | None)

  • data_2_copy (bool)

field mcu: MCU [Required]
field data: Path | Annotated[str, StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=3, max_length=8000000, pattern=None)] [Required]
field data_type: FirmwareDType [Required]
field data_hash: str | None = None
field data_2_copy: bool = True

⤷ signals that file has to be copied to testbed

pydantic model shepherd_core.data_models.content.VirtualHarvesterConfig

The virtual harvester configuration characterizes usage of an energy environment.

It is used to both harvesting during emulation and to record energy environments (sometimes referred to as “harvesting traces”).

For emulation:

The virtual harvester configuration describes how energy from a recorded energy environment is harvested. Typically, the energy environment provides an IV-surface, which is a continuous function in three dimensions: voltage, current, and time. Based on this surface, the emulation can derive the available IV-curve at each point in time. The harvester looks up the current that is available (according to the energy environment) from a given harvesting voltage. The harvesting voltage may be dynamically chosen by the harvester based on the implemented harvesting algorithm, which models different real-world harvesters. For example, a maximum power point tracking harvester may choose a harvesting voltage as a ratio of the open circuit voltage available from the energy environment (or transducer in practice).

The energy environments are encoded not as a three-dimensional function, but as IV tuples over time (sampled at a constant frequency). This originates from the technical implementation when recording the IV-surface, where the recorder provides the IV-curve by measuring the current for a given voltage and ramping the voltage from minimal to maximum.

For harvest-recordings:

An energy environment is fully described by the IV surface, which are IV curves over time. Shepherd approximates this by sampling the current at equidistant steps of a voltage ramp. The VirtualHarvesterConfig is also used to parameterize the recording process, typically, it should be configured to record a full IV surface, as this contains the full information of the energy environment. The postponed harvesting is then performed during emulation.

However, it is also possible to record a “pre-harvested” energy environment by performing the harvesting during recording. This results in a recording containing IV samples over time that represent the harvesting voltage (chosen by the virtual harvester during recording) and the current available from the energy environment for that voltage. Together, these represent the power available for harvesting at the time, and during emulation, this power can be converted by the input stage (boost converter) to charge the energy storage.

Fields:

  • algorithm (shepherd_core.data_models.content.enum_datatypes.HarvestAlgorithmDType)

  • samples_n (int)

  • voltage_mV (float)

  • voltage_min_mV (float)

  • voltage_max_mV (float)

  • current_limit_uA (float)

  • voltage_step_mV (float | None)

  • setpoint_n (float)

  • interval_ms (float)

  • duration_ms (float)

  • rising (bool)

  • enable_linear_extrapolation (bool)

  • wait_cycles (int)

field algorithm: HarvestAlgorithmDType [Required]

The algorithm determines how the harvester chooses the harvesting voltage.

field samples_n: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=8), Le(le=2000)])] = 8

How many IV samples are measured for one IV curve.

The curve is recorded by measuring the el. current available from the transducer at equidistant voltage intervals. These voltages are probed by ramping between voltage_min_mV and voltage_max_mV at samples_n points equally distributed over the voltage range. After setting the voltage, the recorder waits for a short period - allowing the analog frontend and transducer to settle - before recording the harvesting current. This wait duration is influenced by wait_cycles.

Selecting all these parameters is a tradeoff between accuracy of the IV curve (density of IV samples) and measurement duration, hence the time accuracy (density of points) of the IV-surface.

Only applicable to recording, not used in emulation.

Used together with voltage_min_mV, voltage_max_mV, rising, and wait_cycles.

Constraints:

  • ge = 8

  • le = 2000

field voltage_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 2500

The harvesting voltage for constant voltage harvesting.

Additionally, for Perturb-and-Observe MPPT, this defines the voltage at startup.

Constraints:

  • ge = 0

  • le = 5000

field voltage_min_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 0

Minimum voltage recorded for the IV curve.

See samples_n for further details.

In emulation, this can be used to “block” parts of the recorded IV curve and not utilize them in the virtual source. However, this is generally discouraged as it can result in discontinuities in the curve and is not well tested. For emulation, this value ideally corresponds to the value of the recorded energy environment.

Constraints:

  • ge = 0

  • le = 5000

field voltage_max_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 5000

Maximum voltage sampled for the curve.

See voltage_min_mV and samples_n.

Constraints:

  • ge = 0

  • le = 5000

field current_limit_uA: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=50000)])] = 50000

For MPPT VOC, the open circuit voltage is identified as the el. current crosses below this threshold.

During recording it allows to keep trajectory in special region (constant current tracking).

Constraints:

  • ge = 1

  • le = 50000

field voltage_step_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=1000000)])] | None = None

The difference between two adjacent voltage samples.

This value is implicitly derived from the other ramp parameters: (voltage_max_mV - voltage_min_mV) / (samples_n - 1)

field setpoint_n: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=1.0)])] = 0.7

The “Open Circuit Voltage Maximum Power Point Tracker” estimates the MPP by taking a constant fraction defined by this parameter of the open circuit voltage. For example, if the IV curve shows an open circuit voltage of 2V and the setpoint is 0.75, then the harvester selects 1.5 volts as the harvesting voltage.

This value is only relevant when ‘algorithm == mppt_voc’.

Constraints:

  • ge = 0

  • le = 1.0

field interval_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.01), Le(le=1000000)])] = 100

The MPP is repeatedly estimated at fixed intervals defined by this duration.

Note that the energy environment can still change in between MPP estimations, but the harvesting voltage is not updated in between.

This value is relevant for all MPP algorithms. For Perturb and Observe, this value is the wait interval between steps.

When an energy environment is recorded with mppt_opt, the optimal harvester is approximated with a very fast Perturb-Observe algorithm, where this interval should be set to a very small value. When emulating with mppt_opt, this value is not relevant as the emulation simply picks the maximum power point from the IV-curve.

Constraints:

  • ge = 0.01

  • le = 1000000

field duration_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.01), Le(le=1000000)])] = 0.1

The duration of MPP sampling.

While performing an MPP sampling every ‘interval_ms’, the input is disconnected to accurately measure the open circuit voltage.

This value is only relevant for mppt_voc.

Constraints:

  • ge = 0.01

  • le = 1000000

field rising: bool = True

Ramp direction for sampling the IV curve.

When set to true, sampling starts at the minimum voltage and ramps up to the maximum.

See samples_n for further details. Not relevant for emulation.

field enable_linear_extrapolation: bool = True

Because the IV curve is not stored fully in PRU memory but streamed sequentially to the PRU, looking up any IV value at any time is not possible. However, because the precision of the emulation degrades until the relevant IV sample passes by, it can extrapolate the available data to estimate the required IV sample.

Enabling extrapolation can yield a better numeric simulation, especially if the harvesting voltage changes rapidly or the IV surface is steep in relevant regions. For example, when emulating a capacitor diode setup and the current falls at high voltages.

This value is only relevant for emulation.

field wait_cycles: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=100)])] = 1

The wait duration to let the analog frontend settle before taking a measurement.

When recording the energy environment, the voltage is set by the digital-to-analog-converter. This parameter delays the current measurement performed by the analog-to-digital converter to allow the harvesting transducer to settle at the defined voltage.

When recording with IscVoc, wait cycles should be added as the analog changes are more significant.

Not relevant for emulation.

Constraints:

  • ge = 0

  • le = 100

pydantic model shepherd_core.data_models.content.VirtualSourceConfig

The vSrc uses the energy environment (file) for supplying the Target Node.

If not already done, the energy will be harvested and then converted during the experiment.

The converter-stage is software defined and offers: - buck-boost-combinations, - a simple diode + resistor and - an intermediate storage capacitor.

Fields:

  • enable_boost (bool)

  • enable_buck (bool)

  • enable_feedback_to_hrv (bool)

  • interval_startup_delay_drain_ms (float)

  • harvester (shepherd_core.data_models.content.virtual_harvester_config.VirtualHarvesterConfig)

  • V_input_max_mV (float)

  • I_input_max_mA (float)

  • V_input_drop_mV (float)

  • R_input_mOhm (float)

  • storage (shepherd_core.data_models.content.virtual_storage_config.VirtualStorageConfig | None)

  • V_intermediate_enable_output_threshold_mV (float)

  • V_intermediate_disable_output_threshold_mV (float)

  • interval_check_thresholds_ms (float)

  • V_pwr_good_enable_threshold_mV (float)

  • V_pwr_good_disable_threshold_mV (float)

  • immediate_pwr_good_signal (bool)

  • C_output_uF (float)

  • V_output_log_gpio_threshold_mV (float)

  • V_input_boost_threshold_mV (float)

  • V_intermediate_max_mV (float)

  • LUT_input_efficiency (list[list[float]])

  • LUT_input_V_min_log2_uV (int)

  • LUT_input_I_min_log2_nA (int)

  • V_output_mV (float)

  • V_buck_drop_mV (float)

  • LUT_output_efficiency (list[float])

  • LUT_output_I_min_log2_nA (int)

field enable_boost: bool = False

⤷ if false -> v_intermediate = v_input, output-switch-hysteresis is still usable

field enable_buck: bool = False

⤷ if false -> v_output = v_intermediate

field enable_feedback_to_hrv: bool = False

Source can control a cv-harvester for ivsurface. Feedback is essential for some harvesters, i.e. diode-circuitry.

field interval_startup_delay_drain_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 0

⤷ Model begins running, but Target is not draining the storage capacitor until this delay is over.

Constraints:

  • ge = 0

  • le = 10000

field harvester: VirtualHarvesterConfig = VirtualHarvesterConfig({'id': 2206, 'name': 'mppt_opt', 'description': 'Power-Optimum with very fast PO-Variant (harvesting) or special max-pwr-picker (emulator / ivcurve)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True, 'algorithm': 'mppt_opt', 'voltage_step_mV': 1.0, 'interval_ms': 0.01})

⤷ Only active / needed if input is ivsurface.

field V_input_max_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 10000

⤷ Maximum input Voltage [mV] -> will be clipped.

Constraints:

  • ge = 0

  • le = 10000

field I_input_max_mA: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290.0)])] = 4200

⤷ Maximum input Current [mA] -> will be clipped.

Constraints:

  • ge = 0

  • le = 4290.0

field V_input_drop_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 0

⤷ simulate voltage drop for input-diode or LDO.

Constraints:

  • ge = 0

  • le = 4290000.0

field R_input_mOhm: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 0

⤷ resistance only active with disabled boost, range [1 mOhm; 1MOhm]

Constraints:

  • ge = 0

  • le = 4290000.0

field storage: VirtualStorageConfig | None = None

⤷ primary intermediate energy storage between boost- and buck-converter stage. Selecting “None” disables the storage and directly connects input to output.

field V_intermediate_enable_output_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 1

⤷ target gets connected (hysteresis-combo with next value)

Constraints:

  • ge = 0

  • le = 10000

field V_intermediate_disable_output_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 0

⤷ target gets disconnected

Constraints:

  • ge = 0

  • le = 10000

field interval_check_thresholds_ms: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290.0)])] = 0

⤷ some ICs (BQ) check every 64 ms if output should be disconnected

Constraints:

  • ge = 0

  • le = 4290.0

field V_pwr_good_enable_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 2800

pwr-good: target is informed on output-pin (hysteresis) -> reference is the intermediate voltage

Constraints:

  • ge = 0

  • le = 10000

field V_pwr_good_disable_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 2200
Constraints:

  • ge = 0

  • le = 10000

field immediate_pwr_good_signal: bool = True

⤷ 1: activate instant schmitt-trigger, 0: stay in interval for checking thresholds

field C_output_uF: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 1.0

final (always last) stage to compensate undetectable current spikes when enabling power for target

Constraints:

  • ge = 0

  • le = 4290000.0

field V_output_log_gpio_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4290000.0)])] = 1400

⤷ minimum voltage threshold needed to enable recording changes in gpio-bank

Constraints:

  • ge = 0

  • le = 4290000.0

field V_input_boost_threshold_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 0

⤷ minimum input-voltage for the boost converter to work

Constraints:

  • ge = 0

  • le = 10000

field V_intermediate_max_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=10000)])] = 10000

⤷ threshold for shutting off boost converter

Constraints:

  • ge = 0

  • le = 10000

field LUT_input_efficiency: Annotated[list[Annotated[list[Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.0), Le(le=1.0)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=12), MaxLen(max_length=12)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=12), MaxLen(max_length=12)])] = [[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]]

⤷ rows are current -> first row a[V=0][:]

input-LUT[12][12] depending on array[inp_voltage][log(inp_current)], influence of cap-voltage is not implemented

Constraints:

  • min_length = 12

  • max_length = 12

field LUT_input_V_min_log2_uV: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=20)])] = 0

⤷ i.e. 2^7 = 128 uV -> LUT[0][:] is for inputs < 128 uV

Constraints:

  • ge = 0

  • le = 20

field LUT_input_I_min_log2_nA: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=20)])] = 1

⤷ i.e. 2^8 = 256 nA -> LUT[:][0] is for inputs < 256 nA

Constraints:

  • ge = 1

  • le = 20

field V_output_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 2400

Fixed Voltage of Buck-Converter. (as long as Input is > Output + Drop-Voltage)

Constraints:

  • ge = 0

  • le = 5000

field V_buck_drop_mV: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=5000)])] = 0

⤷ simulate LDO / diode minimum voltage differential or output-diode

Constraints:

  • ge = 0

  • le = 5000

field LUT_output_efficiency: Annotated[list[Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.0), Le(le=1.0)])]], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=12), MaxLen(max_length=12)])] = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

⤷ array[12] depending on output_current, In- & Output is linear.

Constraints:

  • min_length = 12

  • max_length = 12

field LUT_output_I_min_log2_nA: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=20)])] = 1

⤷ i.e. 2^8 = 256 nA -> LUT[0] is for inputs < 256 nA, see notes on LUT_input for explanation

Constraints:

  • ge = 1

  • le = 20

Tasks

These are digestible configs for shepherd-herd or -sheep.

Link to Source

pydantic model shepherd_core.data_models.task.HarvestTask

Config for the Observer in Harvest-Mode to record IV data from a harvesting-source.

Fields:

  • output_path (pathlib.Path)

  • force_overwrite (bool)

  • output_compression (shepherd_core.data_models.task.emulation.Compression | None)

  • time_start (datetime.datetime | None)

  • duration (datetime.timedelta | None)

  • use_cal_default (bool)

  • virtual_harvester (shepherd_core.data_models.content.virtual_harvester_config.VirtualHarvesterConfig)

  • power_tracing (shepherd_core.data_models.experiment.observer_features.PowerTracing)

  • sys_logging (shepherd_core.data_models.experiment.observer_features.SystemLogging | None)

  • verbose (int)

field output_path: Path [Required]

⤷ dir- or file-path for storing the recorded data:

  • providing a directory -> file is named hrv_timestamp.h5

  • for a complete path the filename is not changed except it exists and overwrite is disabled -> name#num.h5

field force_overwrite: bool = False

⤷ Overwrite existing file

field output_compression: Compression | None = Compression.gzip1

⤷ should be 1 (level 1 gzip), lzf, or None (order of recommendation)

field time_start: datetime | None = None

timestamp or unix epoch time, None = ASAP

field duration: timedelta | None = None

⤷ Duration of recording in seconds, None = till EOFSys

field use_cal_default: bool = False

⤷ Use default calibration values, skip loading from EEPROM

field virtual_harvester: VirtualHarvesterConfig = VirtualHarvesterConfig({'id': 2206, 'name': 'mppt_opt', 'description': 'Power-Optimum with very fast PO-Variant (harvesting) or special max-pwr-picker (emulator / ivcurve)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True, 'algorithm': 'mppt_opt', 'voltage_step_mV': 1.0, 'interval_ms': 0.01})

⤷ Choose one of the predefined virtual harvesters or configure a new one

field power_tracing: PowerTracing = PowerTracing({})
field sys_logging: SystemLogging | None = SystemLogging({})
field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug

Constraints:

  • ge = 0

  • le = 4

pydantic model shepherd_core.data_models.task.EmulationTask

Configuration for the Observer in Emulation-Mode.

Fields:

  • input_path (pathlib.Path)

  • output_path (pathlib.Path | None)

  • force_overwrite (bool)

  • output_compression (shepherd_core.data_models.task.emulation.Compression | None)

  • time_start (datetime.datetime | None)

  • duration (datetime.timedelta | None)

  • use_cal_default (bool)

  • enable_io (bool)

  • io_port (shepherd_core.data_models.testbed.cape.TargetPort)

  • pwr_port (shepherd_core.data_models.testbed.cape.TargetPort)

  • voltage_aux (float | str)

  • virtual_source (shepherd_core.data_models.content.virtual_source_config.VirtualSourceConfig)

  • power_tracing (shepherd_core.data_models.experiment.observer_features.PowerTracing | None)

  • gpio_tracing (shepherd_core.data_models.experiment.observer_features.GpioTracing | None)

  • uart_logging (shepherd_core.data_models.experiment.observer_features.UartLogging | None)

  • gpio_actuation (shepherd_core.data_models.experiment.observer_features.GpioActuation | None)

  • sys_logging (shepherd_core.data_models.experiment.observer_features.SystemLogging | None)

  • verbose (int)

field input_path: Path [Required]

⤷ hdf5 file containing harvesting data

field output_path: Path | None = None

⤷ dir- or file-path for storing the recorded data:

  • providing a directory -> file is named emu_timestamp.h5

  • for a complete path the filename is not changed except it exists and overwrite is disabled -> emu#num.h5

TODO: should the output-path be mandatory?

field force_overwrite: bool = False

⤷ Overwrite existing file

field output_compression: Compression | None = Compression.gzip1

⤷ should be lzf, 1 (gzip level 1) or None (order of recommendation)

field time_start: datetime | None = None

timestamp or unix epoch time, None = ASAP

field duration: timedelta | None = None

⤷ Duration of recording in seconds, None = till EOF

field use_cal_default: bool = False

⤷ Use default calibration values, skip loading from EEPROM

field enable_io: bool = True

⤷ Switch the GPIO level converter to targets on/off

pre-req for sampling gpio / uart,

field io_port: TargetPort = TargetPort.A

⤷ Either Port A or B that gets connected to IO

field pwr_port: TargetPort = TargetPort.A

⤷ selected port will be current-monitored

  • main channel is nnected to virtual Source

  • the other port is aux

field voltage_aux: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4.5)])] | str = 0

⤷ aux_voltage options - 0-4.5 for specific const Voltage (0 V = disabled), - “buffer” will output intermediate voltage (storage cap of vsource), - “main” will mirror main target voltage

field virtual_source: VirtualSourceConfig = VirtualSourceConfig({'id': 1000, 'name': 'neutral', 'description': 'Direct feed-through of energy environment with no converter (allows on-off-patters)', 'created': datetime.datetime(2022, 12, 12, 12, 12, 12), 'updated_last': datetime.datetime(2022, 12, 12, 12, 12, 12), 'owner': 'Ingmar', 'group': 'NES Lab', 'visible2group': True, 'visible2all': True})

⤷ Use the desired setting for the virtual source,

provide parameters or name like BQ25570

field power_tracing: PowerTracing | None = PowerTracing({})
field gpio_tracing: GpioTracing | None = GpioTracing({})
field uart_logging: UartLogging | None = UartLogging({})
field gpio_actuation: GpioActuation | None = None
field sys_logging: SystemLogging | None = SystemLogging({})
field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug,

TODO: just bool now, systemwide

Constraints:

  • ge = 0

  • le = 4

pydantic model shepherd_core.data_models.task.FirmwareModTask

Config for Task that adds the custom ID to the firmware & stores it into a file.

Fields:

  • data (str | pathlib.Path)

  • data_type (shepherd_core.data_models.content.enum_datatypes.FirmwareDType)

  • custom_id (int | None)

  • firmware_file (pathlib.Path)

  • verbose (int)

field data: Annotated[str, StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=3, max_length=8000000, pattern=None)] | Path [Required]
field data_type: FirmwareDType [Required]
field custom_id: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Lt(lt=65536)])] | None = None
field firmware_file: Path [Required]
field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug

Constraints:

  • ge = 0

  • le = 4

pydantic model shepherd_core.data_models.task.ProgrammingTask

Config for a Task programming the selected target.

Fields:

  • firmware_file (pathlib.Path)

  • target_port (shepherd_core.data_models.testbed.cape.TargetPort)

  • mcu_port (int)

  • mcu_type (str)

  • voltage (float)

  • datarate (int)

  • protocol (shepherd_core.data_models.testbed.mcu.ProgrammerProtocol)

  • simulate (bool)

  • verbose (int)

field firmware_file: Path [Required]
field target_port: TargetPort = TargetPort.A
field mcu_port: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=2)])] = 1
Constraints:

  • ge = 1

  • le = 2

field mcu_type: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern=^[ -~]+$)] [Required]

⤷ must be either “nrf52” or “msp430” ATM, TODO: clean Experiment to tasks

Constraints:

  • pattern = ^[ -~]+$

field voltage: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Lt(lt=5)])] = 3
Constraints:

  • ge = 1

  • lt = 5

field datarate: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Gt(gt=0), Le(le=1000000)])] = 200000
Constraints:

  • gt = 0

  • le = 1000000

field protocol: ProgrammerProtocol [Required]
field simulate: bool = False
field verbose: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=4)])] = 2

⤷ 0=Errors, 1=Warnings, 2=Info, 3=Debug

Constraints:

  • ge = 0

  • le = 4

pydantic model shepherd_core.data_models.task.ObserverTasks

Collection of tasks for selected observer included in experiment.

Fields:

  • observer (str)

  • time_prep (datetime.datetime | None)

  • root_path (pathlib.Path)

  • fw1_mod (shepherd_core.data_models.task.firmware_mod.FirmwareModTask | None)

  • fw2_mod (shepherd_core.data_models.task.firmware_mod.FirmwareModTask | None)

  • fw1_prog (shepherd_core.data_models.task.programming.ProgrammingTask | None)

  • fw2_prog (shepherd_core.data_models.task.programming.ProgrammingTask | None)

  • emulation (shepherd_core.data_models.task.emulation.EmulationTask | None)

  • owner_id (int | None)

  • abort_on_error (bool)

field observer: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=50, pattern=^[^<>:;,?\"\*|\/\\]+$)] [Required]
Constraints:

  • max_length = 50

  • pattern = ^[^<>:;,?"*|/\]+$

field time_prep: datetime | None = None
field root_path: Path [Required]
field fw1_mod: FirmwareModTask | None = None
field fw2_mod: FirmwareModTask | None = None
field fw1_prog: ProgrammingTask | None = None
field fw2_prog: ProgrammingTask | None = None
field emulation: EmulationTask | None = None
pydantic model shepherd_core.data_models.task.TestbedTasks

Collection of tasks for all observers included in experiment.

Fields:

  • name (str)

  • observer_tasks (list[shepherd_core.data_models.task.observer_tasks.ObserverTasks])

field name: StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=50, pattern=^[^<>:;,?\"\*|\/\\]+$)] [Required]
Constraints:

  • max_length = 50

  • pattern = ^[^<>:;,?"*|/\]+$

field observer_tasks: Annotated[list[ObserverTasks], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=128)])] [Required]
Constraints:

  • min_length = 1

  • max_length = 128