Three more drivers on the protocol description, and a format that stays about the wire

CHANGELOG.md

@@ -30,14 +30,34 @@ API changes can land even when the minor version is unchanged.
 - **`ProtocolDebugPort`** (`communication/debugport.py`), a debug port with no code
   of its own: hand it a `CommandDictionary` and it stands in for the instrument.
   Whatever a request carries is remembered by name and whatever a reply carries is
-  answered from that memory. A command may add a `"sets"` clause for a state change
-  its bytes cannot express -- `HOME` carries nothing and still moves a stage.
+  answered from that memory. Two things stay outside it, being facts about the
+  instrument rather than its protocol: what it reads as when switched on, and a
+  command that changes state it does not carry (`HOME` carries nothing and still
+  moves a stage). A device needing either subclasses the port, which is also where
+  anything a value store cannot model belongs -- a laser taking a second to reach
+  a new power. A device needing none of it uses the class as it stands.
 - **`PhysicalDevice.protocol`** (default `None`) and **`PhysicalDevice.performTransaction()`**,
   which performs one command of that protocol under the port's `transactionLock`,
   reading by `readLength` for a binary reply and up to the terminator for a line.
   A driver that sets `protocol` needs no send-and-receive code of its own.
 
 ### Changed
+- **`EchoDevice`, `IntegraDevice` and `CoboltDevice` speak through a description**,
+  as `SutterDevice` does. Each keeps its protocol as data and sends through
+  `performTransaction`; `communication/commands.py` now has one user left,
+  `IntellidriveDevice`.
+  - **`IntegraDevice` gains a debug path**: `IntegraDevice("debug")` builds a
+    `ProtocolDebugPort`. It had none, so every one of its tests skipped unless the
+    meter was plugged in; five now run without it.
+  - `EchoDevice`'s three commands were **timing out and saying nothing about it**.
+    Its replies carry no terminator, so reading a line ran to the end of the
+    buffer, and the exception was swallowed into an attribute -- which is why
+    `testEchoCommands` passed. Described as fixed-size frames whose every byte is
+    a constant, an echo that comes back wrong is now refused.
+  - `CoboltDevice`'s debug port never updated `requestedPower`, so
+    `GET_REQUESTED_POWER` always answered 0. On a Cobolt, `p` sets the power asked
+    for and `pa?` reads the power reached; the description now names those two
+    apart, and both read correctly.
 - **`SutterDevice` speaks through a description.** Its protocol is data, it sends
   through `performTransaction`, and it no longer checks acknowledgements by hand:
   the description says `MOVE`, `HOME` and `WORK` answer `b"\r"`. Its nested

hardwarelibrary/communication/debugport.py

@@ -180,12 +180,12 @@ class ProtocolDebugPort(DebugPort):
     just been switched on. A reply that cannot pack a 0 -- one carrying text, say
     -- will say so rather than invent something.
 
-    The one thing no description of bytes can express is a command that changes
-    the instrument without carrying anything: HOME takes no arguments and yet
-    moves the stage to its origin. A command may therefore state a "sets" clause,
-    which is applied on arrival exactly as a request's own values are. That
-    clause is the only part of a description that talks about the instrument
-    rather than the wire, and a driver never reads it.
+    Two things are deliberately outside it, because they are facts about the
+    instrument and not about its protocol: what it reads as when switched on, and
+    a command that changes state it does not carry -- HOME takes no arguments and
+    still moves a stage. Both belong to a subclass, which is also where anything
+    a store cannot model belongs, such as a laser taking a second to reach a new
+    power. A device needing none of that uses this class as it stands.
 
     Recognising a request is the dictionary's work, not this class's: recognize()
     asks each command in turn, and the header constants settle it. So there is no
@@ -223,7 +223,6 @@ def processInputBuffers(self, endPointIndex):
         self.inputBuffers[endPointIndex] = bytearray()
 
         self.values.update(arguments)
-        self.values.update(command.sets)
         if command.expectsReply:
             self.writeToOutputBuffer(command.encodeReply(**self.answerFor(command)),
                                      endPointIndex)

hardwarelibrary/communication/protocol.py

@@ -1,11 +1,13 @@
 """Describing an instrument protocol, without performing it.
 
-The idea is sans-I/O, the principle behind h11 and wsproto: a protocol is a pure
-transformation over bytes and never performs the exchange. A Frame turns named
-values into the bytes to write and the bytes read back into named values; it owns
-no port and stores nothing of what happened. A driver keeps the port and calls the
-primitives of CommunicationPort itself, so the same description serves a driver
-speaking to hardware and a debug port standing in for it.
+The idea is simple: describe the protocol of a device in a way general enough
+to create a debug port to test it without hardware and minimize the details
+of sending the commands. A protocol manipulates the bytes and formats them,
+but never performs the send or the read. A Frame turns named values into
+the bytes to write and the bytes read back into named values; it owns no port
+and stores nothing of what happened. A driver keeps the port and calls the
+primitives of CommunicationPort itself, so the same description serves a
+driver speaking to hardware and a debug port standing in for it.
 
 Consequences worth noticing while reading:
 
@@ -658,7 +660,7 @@ class Command:
     """
 
     def __init__(self, name: str, request: Frame, reply: Optional[Frame] = None,
-                 sets: Optional[dict] = None, specimen: Optional[dict] = None):
+                 specimen: Optional[dict] = None):
         """Pair a request with the reply it expects, under the name a driver uses.
 
         Args:
@@ -667,23 +669,14 @@ def __init__(self, name: str, request: Frame, reply: Optional[Frame] = None,
             request: the frame the driver writes and a mock reads
             reply: the frame the driver reads and a mock writes, or None for a
                 command the instrument does not answer
-            sets: what receiving this command does to the instrument's state,
-                as {field name: value}, for what the request does not carry.
-                HOME takes no arguments and yet moves the stage to the origin,
-                and nothing about the bytes on the wire could say so. This is the
-                one thing here that describes the instrument rather than the
-                protocol, and it exists so that a debug port needs no code of its
-                own; a driver never reads it.
             specimen: a value per field for validate() to send through this
                 command and expect back, when the ones it makes from the declared
                 types will not do -- an expression that refuses a zero, say.
-                Like sets, it is not part of the protocol, and only validate()
-                reads it.
+                It is not part of the protocol, and only validate() reads it.
         """
         self.name = name
         self.request = request
         self.reply = reply
-        self.sets = dict(sets or {})
         self.specimen = dict(specimen or {})
 
     @property
@@ -976,8 +969,7 @@ def commandFrom(cls, name: str, description: dict) -> Command:
             name: the command's name, used both for the Command and to say which
                 command an error is about
             description: {"request": {...}} and, when the instrument answers,
-                {"reply": {...}}, plus an optional {"sets": {...}} for a state
-                change the request does not carry
+                {"reply": {...}}
 
         Returns:
             The command, its reply None when none was described.
@@ -992,7 +984,6 @@ def commandFrom(cls, name: str, description: dict) -> Command:
         reply = description.get("reply")
         return Command(name, cls.requestFrom(description["request"], where),
                        cls.replyFrom(reply, where) if reply is not None else None,
-                       sets=description.get("sets"),
                        specimen=description.get("specimen"))
 
     @classmethod

hardwarelibrary/echodevice.py

@@ -1,46 +1,72 @@
-from hardwarelibrary.physicaldevice import *
-from hardwarelibrary.communication.communicationport import *
-from hardwarelibrary.communication.serialport import *
-from hardwarelibrary.communication.commands import DataCommand, DataDecoder, TextCommand
-from hardwarelibrary.communication.debugport import TableDrivenDebugPort
+from hardwarelibrary.communication.debugport import ProtocolDebugPort
+from hardwarelibrary.communication.protocol import CommandDictionary
+from hardwarelibrary.communication.serialport import SerialPort
+from hardwarelibrary.physicaldevice import PhysicalDevice
 
-import re
-import time
-from struct import *
+
+# An echo sends its payload back verbatim, with nothing around it. There is no
+# terminator to read up to, so a reply is a fixed-size frame -- and since the
+# whole content is known in advance, every byte of it is a constant. That is what
+# turns "we received something" into "we received exactly what we sent": a
+# constant is required on the way in, so a reply that differs is refused.
+echoProtocol = {
+    "device": "FTDI echo device",
+    "commands": {
+        "ECHO1": {
+            "request": {"struct": "<8s", "fields": ["text"],
+                        "constants": {"text": "someText"}},
+            "reply": {"struct": "<8s", "fields": ["text"],
+                      "constants": {"text": "someText"}},
+        },
+        "ECHO2": {
+            "request": {"struct": "<13s", "fields": ["text"],
+                        "constants": {"text": "someOtherText"}},
+            "reply": {"struct": "<13s", "fields": ["text"],
+                      "constants": {"text": "someOtherText"}},
+        },
+        "ECHO3": {
+            "request": {"struct": "<8s", "fields": ["data"],
+                        "constants": {"data": "someData"}},
+            "reply": {"struct": "<8s", "fields": ["data"],
+                      "constants": {"data": "someData"}},
+        },
+    },
+}
 
 
 class EchoDevice(PhysicalDevice):
+    """A device that sends back whatever it is given, over an FTDI cable.
+
+    It exists to exercise the machinery rather than to measure anything, so its
+    protocol is three payloads that must come back unchanged.
+    """
+
     classIdProduct = 0x6001
     classIdVendor = 0x0403
     usesGenericSerialConverter = True
-    commands = {
-        "ECHO1": TextCommand(name="ECHO1", requestEncoder="someText", replyDecoder="someText"),
-        "ECHO2": TextCommand(name="ECHO2", requestEncoder="someOtherText", replyDecoder="someOtherText"),
-        "ECHO3": DataCommand(name="ECHO3", data=b"someData",
-                             replyDecoder=DataDecoder(length=len(b"someData"))),
-    }
 
-    def __init__(self, serialNumber='ftDXIKC4', idProduct=classIdProduct, idVendor=classIdVendor):
-        PhysicalDevice.__init__(self, serialNumber=serialNumber, idProduct=idProduct, idVendor=idVendor)
+    protocol = CommandDictionary.fromDescription(echoProtocol)
+
+    def __init__(self, serialNumber='ftDXIKC4', idProduct=classIdProduct,
+                 idVendor=classIdVendor):
+        """Bind to one FTDI cable, or to "debug" for a port that echoes in memory.
+
+        Args:
+            serialNumber: the cable's serial number, or "debug"
+            idProduct: USB product id, defaulting to the class attribute
+            idVendor: USB vendor id, defaulting to the class attribute
+        """
+        PhysicalDevice.__init__(self, serialNumber=serialNumber,
+                                idProduct=idProduct, idVendor=idVendor)
 
     def doInitializeDevice(self):
+        """Open the cable, or stand one up out of the description."""
         if self.serialNumber == "debug":
-            self.port = self.DebugSerialPort()
+            self.port = ProtocolDebugPort(self.protocol)
         else:
             self.port = SerialPort(idVendor=self.idVendor, idProduct=self.idProduct)
         self.port.open()
 
     def doShutdownDevice(self):
+        """Close the port."""
         self.port.close()
-
-    class DebugSerialPort(TableDrivenDebugPort):
-        def __init__(self):
-            super().__init__(commands=EchoDevice.commands)
-
-        def process_command(self, name, params, endPointIndex):
-            if name == 'ECHO1':
-                return 'someText'
-            elif name == 'ECHO2':
-                return 'someOtherText'
-            elif name == 'ECHO3':
-                return b'someData'

hardwarelibrary/motion/sutterdevice.py

@@ -34,9 +34,6 @@
                         "constants": {"header": "H", "terminator": "\r"}},
             "reply": {"struct": "<c", "fields": ["acknowledgement"],
                       "constants": {"acknowledgement": "\r"}},
-            # The one thing the bytes cannot say: HOME carries nothing and yet
-            # moves the stage to the origin. Only a debug port reads this.
-            "sets": {"x": 0, "y": 0, "z": 0},
         },
         "WORK": {
             "request": {"struct": "<cc", "fields": ["header", "terminator"],
@@ -102,7 +99,7 @@ def doInitializeDevice(self):
         """
         try:
             if self.serialNumber == "debug":
-                self.port = ProtocolDebugPort(self.protocol)
+                self.port = self.DebugSerialPort(self.protocol)
                 self.port.open()
             else:
                 portPath = SerialPort.matchAnyPort(idVendor=self.idVendor,
@@ -183,3 +180,24 @@ def work(self):
         """Send the stage home, then to its work position."""
         self.home()
         self.performTransaction("WORK")
+
+    class DebugSerialPort(ProtocolDebugPort):
+        """An MP-285 without an MP-285, out of its own description.
+
+        The description carries the whole protocol, so the only thing left here is
+        the one fact its bytes cannot state: HOME carries no arguments and still
+        sends the stage to the origin.
+        """
+
+        def answerFor(self, command) -> dict:
+            """Move to the origin on HOME, then answer as the description says.
+
+            Args:
+                command: the command that was recognized
+
+            Returns:
+                The values its reply carries, taken from what is remembered.
+            """
+            if command.name == "HOME":
+                self.values.update(x=0, y=0, z=0)
+            return super().answerFor(command)