Measurement Component

Learn how to build a custom Measurement and how to get started analyzing the data.

We will describe how to build the number_gen_readout measurement that works together with the ScopeFoundryHW package we made in the previous tutorial. When run, this measurement periodically samples values from the number_gen hardware component.

Essential Components

A ScopeFoundry Measurement is defined as a subclass of ScopeFoundry.Measurement and has a name:

import time

import numpy as np

from ScopeFoundry import Measurement, h5_io

class NumberGenReadoutSimple(Measurement):

    name = "number_gen_readout_simple"

Then we override the setup() and run() functions that define the measurement. Starting with:

    def setup(self):
        """
        Runs once during App initialization.
        This is the place to load a user interface file,
        define settings, and set up data structures.
        """

        s = self.settings
        s.New("sampling_period", float, initial=0.1, unit="s")
        s.New("N", int, initial=101)
        s.New("save_h5", bool, initial=True)

This defines 3 parameters that will be used during the measurement.

When a measurement is started, a new thread is launched, within which eventually the run() function is called. Let’s override it to:

  1. Sample values from the “number_gen” hardware component:
    def run(self):
        """
        Runs when measurement is started. Runs in a separate thread from the GUI.
        It should not update the graphical interface directly and should only
        focus on data acquisition.
        """
        # Prepare an array for data in memory.
        self.data = np.ones(self.settings["N"])
        
        # Get a reference to the hardware
        self.hw = self.app.hardware["number_gen"]
        
        # N-times sampling the hardware for values
        for i in range(self.settings["N"]):
            self.data[i] = self.hw.settings.sine_data.read_from_hardware()
            time.sleep(self.settings["sampling_period"])
            self.set_progress(i * 100.0 / self.settings["N"])
            if self.interrupt_measurement_called:
                break
  • The interrupt_measurement_called flag is set to True when the user stops the measurement. Here it breaks out of the loop as the measurement spends most of its time there.
  • Using set_progress(), the progress bar is updated, and an estimated time until the measurement is done is calculated based on the time it started and the progress percentage you set.
  1. Save it to an HDF5 file (if the user desires). With this boilerplate code, all settings from every hardware and the measurement are saved.
        if self.settings["save_h5"]:
            # Open a file
            self.h5_file = h5_io.h5_base_file(app=self.app, measurement=self)

            # Create a measurement H5 group (folder) within self.h5file
            # This stores all the measurement metadata in this group
            self.h5_group = h5_io.h5_create_measurement_group(
                measurement=self, h5group=self.h5_file
            )

            # Dump the dataset and close the file.
            self.h5_group.create_dataset(name="y", data=self.data)
            self.h5_file.close()

The Case for Using self.settings

  • When saving data as written above, the values are added to the resulting file, which is useful:
    • To analyze data.
    • The user can drag and drop the file on the app to reload the values and bring ScopeFoundry to the same state.
  • ScopeFoundry already generates widgets in the left tree that the user can use to set values.
  • Provides a coherent way to access settings in other components. For example, here we referenced a setting from the “number_gen” hardware component, asked it to update itself, and retrieved a value.
  • An easy way to generate a GUI and connect to widgets in GUIs, as you will see next.

Adding a Graphical User Interface

We use two Qt-based libraries to create the UI. Let’s import them at the top of the file:

import pyqtgraph as pg
from qtpy import QtCore, QtWidgets

The GUI should be created at startup. Hence, override the setup_figure function (which gets called after the setup function). ScopeFoundry expects that setup_figure defines self.ui with a widget.

Here we define the GUI programmatically (alternatively, one can use Qt Creator, see below):

    def setup_figure(self):
        self.ui = QtWidgets.QWidget()

QtWidgets.QWidget() is an empty widget.

To add widgets onto self.ui, one must use a layout. (In the Qt world, one cannot add widgets directly onto a widget.)

        layout = QtWidgets.QVBoxLayout()
        self.ui.setLayout(layout)

The type of layout defines how added widgets are arranged. Here, QVBoxLayout stacks them vertically. ScopeFoundry provides convenience methods to create widgets that, out of the box, update when settings values change and conversely change the settings value when its corresponding widget is changed. Let’s add widgets for the settings defined in the setup function and a start/stop button to the layout:

        layout.addWidget(self.settings.New_UI(include=("sampling_period", "N", "save_h5")))
        layout.addWidget(self.new_start_stop_button())

Finally, let’s add the plot widget, with axes and a line:

        self.graphics_widget = pg.GraphicsLayoutWidget(border=(100, 100, 100))
        self.plot = self.graphics_widget.addPlot(title=self.name)
        self.plot_lines = {}
        self.plot_lines["y"] = self.plot.plot(pen="g")
        layout.addWidget(self.graphics_widget)

ScopeFoundry calls update_display() repeatedly during a measurement. Let’s override it:

    def update_display(self):
        self.plot_lines["y"].setData(self.data["y"])

Note: You do not have to call update_display yourself. You can control the frequency it gets called with the self.display_update_period attribute.

Putting everything together

We place a number_gen_readout_simple.py next to the fancy_app.py.

# number_gen_readout_simple.py
import time

import numpy as np
import pyqtgraph as pg
from qtpy import QtCore, QtWidgets

from ScopeFoundry import Measurement, h5_io


class NumberGenReadoutSimple(Measurement):

    name = "number_gen_readout_simple"

    def setup(self):
        """
        Runs once during App initialization.
        This is the place to load a user interface file,
        define settings, and set up data structures.
        """

        s = self.settings
        s.New("sampling_period", float, initial=0.1, unit="s")
        s.New("N", int, initial=101)
        s.New("save_h5", bool, initial=True)

    def run(self):
        """
        Runs when the measurement is started. Runs in a separate thread from the GUI.
        It should not update the graphical interface directly and should only
        focus on data acquisition.
        """

        # Prepare an array for data in memory.
        self.data = np.ones(self.settings["N"])
        
        # Get a reference to the hardware
        self.hw = self.app.hardware["number_gen"]
        
        # N-times sampling the hardware for values
        for i in range(self.settings["N"]):
            self.data[i] = self.hw.settings.sine_data.read_from_hardware()
            time.sleep(self.settings["sampling_period"])
            self.set_progress(i * 100.0 / self.settings["N"])
            if self.interrupt_measurement_called:
                break
                        
        if self.settings["save_h5"]:
            # Open a file
            self.h5_file = h5_io.h5_base_file(app=self.app, measurement=self)

            # Create a measurement H5 group (folder) within self.h5file
            # This stores all the measurement metadata in this group
            self.h5_group = h5_io.h5_create_measurement_group(
                measurement=self, h5group=self.h5_file
            )

            # Dump the dataset and close the file
            self.h5_group.create_dataset(name="y", data=self.data)
            self.h5_file.close()
        
    def setup_figure(self):
        """
        Runs once during App initialization and is responsible
        for creating the widget self.ui.        
        """
        self.ui = QtWidgets.QWidget()

        layout = QtWidgets.QVBoxLayout()
        self.ui.setLayout(layout)
        layout.addWidget(self.settings.New_UI(include=("sampling_period", "N", "save_h5")))
        layout.addWidget(self.new_start_stop_button())
        self.graphics_widget = pg.GraphicsLayoutWidget(border=(100, 100, 100))
        self.plot = self.graphics_widget.addPlot(title=self.name)
        self.plot_lines = {"y": self.plot.plot(pen="g")}
        layout.addWidget(self.graphics_widget)

    def update_display(self):
        """
        Updates the display with the latest data.
        """
        self.plot_lines["y"].setData(self.data)

We add this Measurement to the app using the add_measurement() method:

# fancy_app.py
import sys

from ScopeFoundry import BaseMicroscopeApp

class FancyApp(BaseMicroscopeApp):

    name = "fancy app"

    def setup(self):

        from ScopeFoundryHW.random_number_gen import NumberGenHw
        self.add_hardware(NumberGenHw(self))

        from number_gen_readout_simple import NumberGenReadoutSimple
        self.add_measurement(NumberGenReadoutSimple(self))

if __name__ == "__main__":
    app = FancyApp(sys.argv)
    # app.settings_load_ini("default_settings.ini")
    sys.exit(app.exec_())

As usual, this can be run with:

python fancy_app.py

done_after

Next Steps

Bonus: Build the User Interface with Qt Creator

In the above implementation, we created the figure programmatically. However, we could also use Qt Creator to design a user interface.

  1. Download the free Qt Creator.
  2. Create a .ui file. The one used here, number_gen_readout.ui, can be found in the tutorial repository.
    qt-creator-sine-plot-ui
  3. Save the .ui file next to the measurement file (sibling path).
  4. Adjust the setup_figure() method of the measurement:
    def setup_figure(self):
        """
        Runs once during App initialization, after setup().
        This is the place to make all graphical interface initializations,
        build plots, etc.
        """
        self.ui_filename = sibling_path(__file__, "number_gen_readout.ui")
        self.ui = load_qt_ui_file(self.ui_filename)

        # Connect UI widgets to measurement/hardware settings or functions
        self.settings.activation.connect_to_pushButton(self.ui.start_pushButton)
        self.settings.save_h5.connect_to_widget(self.ui.save_h5_checkBox)
        self.hw.settings.amplitude.connect_to_widget(self.ui.amp_doubleSpinBox)
        
        # Set up pyqtgraph graph_layout in the UI
        self.graph_layout = pg.GraphicsLayoutWidget()
        self.ui.plot_groupBox.layout().addWidget(self.graph_layout)

        # Create PlotItem object (a set of axes)  
        self.plot = self.graph_layout.addPlot(title=self.name)
        # Create PlotDataItem object (a scatter plot on the axes)
        self.plot_lines = {"y": self.plot.plot(pen="g")}

The resulting app should look like:

microscope-with-func-gen

Bonus 2: Improved Version

In the above example, we kept things simple. We made some modifications in this final version that has the following improvements:

  1. run():
    • The measurement runs indefinitely or until the user hits stop.
    • Data is dumped to the file during the measurement, ensuring that data is stored if the program crashes.
  2. setup_figure():
    • Uses a splitter instead of QVBoxLayout.
    • Includes settings from the hardware.
# number_gen_readout.py
import time

import numpy as np
import pyqtgraph as pg
from qtpy import QtCore, QtWidgets

from ScopeFoundry import Measurement, h5_io


class NumberGenReadout(Measurement):

    name = "number_gen_readout"

    def setup(self):
        """
        Runs once during App initialization.
        This is the place to load a user interface file,
        define settings, and set up data structures.
        """

        s = self.settings
        s.New("sampling_period", float, initial=0.1, unit="s")
        s.New("N", int, initial=101)
        s.New("save_h5", bool, initial=True)

        # Data structure of the measurement
        self.data = {"y": np.ones(101)}
        
        # Link to previous functions
        self.hw = self.app.hardware["number_gen"]

    def setup_figure(self):
        """
        Runs once during App initialization and is responsible
        for creating the widget self.ui.        
        
        Here we create the UI figure programmatically. For an alternative using Qt 
        Creator, see below.
        """

        # Make a layout that holds all measurement controls and settings from hardware
        cb_layout = QtWidgets.QHBoxLayout()
        cb_layout.addWidget(self.new_start_stop_button())
        cb_layout.addWidget(
            self.settings.New_UI(
                exclude=("activation", "run_state", "profile", "progress")
            )
        )
        # Add hardware settings to the layout
        cb_layout.addWidget(self.hw.settings.New_UI(exclude=("debug_mode", "connected", "port")))
        header_widget = QtWidgets.QWidget()
        header_layout = QtWidgets.QVBoxLayout(header_widget)
        header_layout.addLayout(cb_layout)

        # Make a plot widget containing one line
        self.graphics_widget = pg.GraphicsLayoutWidget(border=(100, 100, 100))
        self.plot = self.graphics_widget.addPlot(title=self.name)
        self.plot_lines = {}
        self.plot_lines["y"] = self.plot.plot(pen="g")

        # Putting everything together
        # ScopeFoundry assumes .ui is the main widget:
        self.ui = QtWidgets.QSplitter(QtCore.Qt.Orientation.Vertical)
        self.ui.addWidget(header_widget)
        self.ui.addWidget(self.graphics_widget)

    def setup_h5_file(self):
        # This stores all the hardware and app metadata in the H5 file
        self.h5file = h5_io.h5_base_file(app=self.app, measurement=self)

        # Create a measurement H5 group (folder) within self.h5file
        # This stores all the measurement metadata in this group
        self.h5_group = h5_io.h5_create_measurement_group(
            measurement=self, h5group=self.h5file
        )

        # Create an H5 dataset to store the data
        dset = self.data["y"]
        self.h5_y = self.h5_group.create_dataset(
            name="y", shape=dset.shape, dtype=dset.dtype
        )

    def run(self):
        """
        Runs when the measurement is started. Runs in a separate thread from the GUI.
        It should not update the graphical interface directly and should only
        focus on data acquisition.
        """

        # A buffer in memory for data
        self.data["y"] = np.ones(self.settings["N"])

        if self.settings["save_h5"]:
            self.setup_h5_file()

        # Use a try/finally block to ensure cleanup
        try:
            i = 0

            # Will run forever until interrupt is called
            while not self.interrupt_measurement_called:
                i %= len(self.h5_y)

                # Set progress bar percentage complete
                self.set_progress(i * 100.0 / self.settings["N"])

                # Fill the buffer with sine wave readings from func_gen hardware
                self.data["y"][i] = self.hw.settings.sine_data.read_from_hardware()

                if self.settings["save_h5"]:
                    # If saving data to disk, copy data to H5 dataset
                    self.h5_y[i] = self.data["y"][i]
                    # Flush H5
                    self.h5file.flush()

                # Wait between readings
                time.sleep(self.settings["sampling_period"])

                i += 1

        finally:
            print("NumberGenReadout: Finishing")
            if self.settings["save_h5"]:
                # Make sure to close the data file
                self.h5file.close()
                
    def update_display(self):
        """
        Updates the display with the latest data.
        """
        self.plot_lines["y"].setData(self.data["y"])

Result of Improved Version:

done_after

Next Steps