Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
- name: Publish package to PyPI
run: uv publish
- name: upload to github release
uses: docker://antonyurchenko/git-release:v5
uses: docker://antonyurchenko/git-release:v7
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# assume any version with "a" (gets "ALPHA") or "b" (gets BETA) or "rc" (release candidates) will be a prerelease
Expand Down
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,26 @@

We follow [Common Changelog](https://common-changelog.org/) formatting for this document.

## Unreleased

### Changed

- Replaced `has_error` field in `UserspaceMessage` with `message_state`. Instead of a boolean, this is an enumerated value. This enumerated value can capture ongoing processing in addition to error notifications. ([commit](https://github.com/INTERSECT-SDK/intersect-python-common/commit/8c144b2186228cbcef44edf256a3a81bd101d9f1) (Lance Drane))
- Consistently use '/' character everywhere to indicate a separator character, only change this in the protocol layer as needed ([commit](https://github.com/INTERSECT-SDK/intersect-python-common/commit/038725cf18a6b22ba6f24ccebec389f78e25b49f) (Lance Drane))

### Added

- Added `IntersectConfig` as representative of the configuration retrieved from the Registry Service at runtime ([commit 1](https://github.com/INTERSECT-SDK/intersect-python-common/commit/d4167449cca8fa05838379bc875c55876552d229) [commit 2](https://github.com/INTERSECT-SDK/intersect-python-common/commit/6e8053ee9555793b84bb2f4e0558c5a7df0ca0e1) (Lance Drane))
- Added mechanism for registry service or other utilities to provide credentials ([commit](https://github.com/INTERSECT-SDK/intersect-python-common/commit/0554cdb6c31bb80cd0e6fc762944e15b7df6fbc6) (Lance Drane))

### Removed

- HierarchyConfig has been removed entirely. We no longer use organization, facility, or subsystem anywhere across the ecosystem. `System` is preserved as a top-level namespace to be configured by the ecosystem, and `Service` is the namespace the user/client specifies. ([commit](https://github.com/INTERSECT-SDK/intersect-python-common/commit/d4167449cca8fa05838379bc875c55876552d229) (Lance Drane))

### Fixed

- Force ControlPlaneManager `disconnect()` command to stop hanging ([commit](https://github.com/INTERSECT-SDK/intersect-python-common/commit/6686d744af9bc38686a0fb5baf53270cd0308080) (Lance Drane))

## [0.9.6] - 2026-06-02

### Added
Expand Down
130 changes: 61 additions & 69 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,15 @@ readme = "README.md"
license = { text = "BSD-3-Clause" }
requires-python = ">=3.10,<4.0"
keywords = ["intersect"]
version = "0.9.6"
version = "0.10.0a3"
classifiers = [
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
dependencies = [
"pydantic>=2.7.0",
"paho-mqtt>=2.1.0,<3.0.0",
"pika>=1.3.2,<2.0.0",
"minio>=7.2.3",
]
dependencies = ["pydantic>=2.7.0", "paho-mqtt>=2.1.0,<3.0.0", "pika>=1.3.2,<2.0.0", "minio>=7.2.3"]

[project.urls]
Homepage = "https://github.com/INTERSECT-SDK/intersect-python-common"
Expand Down Expand Up @@ -66,88 +61,85 @@ mccabe = { max-complexity = 20 }
pylint = { max-args = 10, max-branches = 20, max-returns = 15, max-statements = 75 }
# pyflakes and the relevant pycodestyle rules are already configured
extend-select = [
'C90', # mccabe complexity
'I', # isort
'N', # pep8-naming
'D', # pydocstyle
'UP', # pyupgrade
'YTT', # flake8-2020
'ANN', # flake8-annotations
'C90', # mccabe complexity
'I', # isort
'N', # pep8-naming
'D', # pydocstyle
'UP', # pyupgrade
'YTT', # flake8-2020
'ANN', # flake8-annotations
'ASYNC', # flake8-async
'S', # flake8-bandit
'BLE', # flake8-blind-except
'B', # flake8-bugbear
'A', # flake8-builtins
'COM', # flake8-commas
'C4', # flake8-comprehensions
'DTZ', # flake8-datetimez
'T10', # flake8-debugger
'EM', # flake8-error-message
'FA', # flake8-future-annotations
'ISC', # flake8-implicit-string-concat
'ICN', # flake8-import-conventions
'G', # flake8-logging-format
'INP', # flake8-no-pep420
'PIE', # flake8-PIE
'T20', # flake8-T20
'PYI', # flake8-pyi
'PT', # flake8-pytest-style
'Q', # flake8-quotes
'RSE', # flake8-raise
'RET', # flake8-return
'SLF', # flake8-self
'SLOT', # flake8-slots
'SIM', # flake8-simplify
'TC', # flake8-type-checking
'ARG', # flake8-unused-arguments
'PTH', # flake8-use-pathlib
'PGH', # pygrep-hooks
'PL', # pylint
'TRY', # tryceratops
'FLY', # flynt
'RUF', # RUFF additional rules
'INT', # flake8-gettext
'S', # flake8-bandit
'BLE', # flake8-blind-except
'B', # flake8-bugbear
'A', # flake8-builtins
'COM', # flake8-commas
'C4', # flake8-comprehensions
'DTZ', # flake8-datetimez
'T10', # flake8-debugger
'EM', # flake8-error-message
'FA', # flake8-future-annotations
'ISC', # flake8-implicit-string-concat
'ICN', # flake8-import-conventions
'G', # flake8-logging-format
'INP', # flake8-no-pep420
'PIE', # flake8-PIE
'T20', # flake8-T20
'PYI', # flake8-pyi
'PT', # flake8-pytest-style
'Q', # flake8-quotes
'RSE', # flake8-raise
'RET', # flake8-return
'SLF', # flake8-self
'SLOT', # flake8-slots
'SIM', # flake8-simplify
'TC', # flake8-type-checking
'ARG', # flake8-unused-arguments
'PTH', # flake8-use-pathlib
'PGH', # pygrep-hooks
'PL', # pylint
'TRY', # tryceratops
'FLY', # flynt
'RUF', # RUFF additional rules
'INT', # flake8-gettext
]
# If you're seeking to disable a rule, first consider whether the rule is overbearing, or if it should only be turned off for your usecase.
ignore = [
'COM812', # formatter, handled by Ruff format
'ISC001', # formatter, handled by Ruff format
'SIM105', # "with contextlib.suppress():" is slower than try-except-pass
'ANN401', # allow explicit "Any" typing, use with care
'COM812', # formatter, handled by Ruff format
'ISC001', # formatter, handled by Ruff format
'SIM105', # "with contextlib.suppress():" is slower than try-except-pass
'ANN401', # allow explicit "Any" typing, use with care
'PLR2004', # allow "magic numbers"
]

[tool.ruff.lint.flake8-type-checking]
runtime-evaluated-base-classes = ["pydantic.BaseModel"]
runtime-evaluated-decorators = [
"pydantic.dataclasses.dataclass",
"pydantic.validate_call",
]
runtime-evaluated-decorators = ["pydantic.dataclasses.dataclass", "pydantic.validate_call"]

[tool.ruff.lint.extend-per-file-ignores]
'__init__.py' = [
'F401', # __init__.py commonly has unused imports
'F401', # __init__.py commonly has unused imports
'TC004', # do lazy imports when importing from the base module
]
'docs/*' = [
'D', # the documentation folder does not need documentation
'D', # the documentation folder does not need documentation
'INP001', # docs are not a namespace package
]
'tests/*' = [
'S101', # allow assert statements in tests
'S106', # don't care about credentials in tests
'S311', # don't care about cryptographic security in tests
'S101', # allow assert statements in tests
'S106', # don't care about credentials in tests
'S311', # don't care about cryptographic security in tests
'SLF001', # allow private member access in tests
'ANN', # tests in general don't need types, unless they are runtime types.
'ARG', # allow unused parameters in tests
'D', # ignore documentation in tests
'ANN', # tests in general don't need types, unless they are runtime types.
'ARG', # allow unused parameters in tests
'D', # ignore documentation in tests
'RUF012', # permit "mutable" class attributes to not be annotated with typing.ClassVar (these shouldn't be mutated anyways...)
]

# see https://mypy.readthedocs.io/en/stable/config_file.html for a complete reference
[tool.mypy]
strict = true
ignore_missing_imports = true # don't require typing for library stubs if they don't exist
ignore_missing_imports = true # don't require typing for library stubs if they don't exist
disallow_untyped_decorators = false # this is needed for library decorator compatibility, i.e. "retrying"
plugins = ["pydantic.mypy"]

Expand All @@ -166,12 +158,12 @@ omit = [
'*__init__*', # __init__ files should just re-export other classes and functions
]
exclude_also = [
'pragma: no-cover', # standard
'pragma: no-cover', # standard
'if (typing\\.)?TYPE_CHECKING:', # type checking blocks are not executed in coverage, but we don't care
'@(abc\\.)?abstractmethod', # don't try to cover abstract methods
"class .*\\bProtocol\\):", # don't cover protocol classes (similar to abstract classes)
'raise NotImplementedError', # it's not implemented so shouldn't be covered
'except.* ImportError', # these are usually used to throw a "friendlier" error and are not really worth testing
'@(abc\\.)?abstractmethod', # don't try to cover abstract methods
"class .*\\bProtocol\\):", # don't cover protocol classes (similar to abstract classes)
'raise NotImplementedError', # it's not implemented so shouldn't be covered
'except.* ImportError', # these are usually used to throw a "friendlier" error and are not really worth testing
]


Expand Down
6 changes: 3 additions & 3 deletions src/intersect_sdk_common/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
ControlProvider,
DataStoreConfig,
DataStoreConfigMap,
HierarchyConfig,
IntersectConfig,
)
from .control_plane.control_plane_manager import ControlPlaneManager
from .control_plane.definitions import MessageCallback
Expand All @@ -40,8 +40,8 @@
'DataPlaneManager',
'DataStoreConfig',
'DataStoreConfigMap',
'HierarchyConfig',
'IntersectApplicationError',
'IntersectConfig',
'IntersectDataHandler',
'IntersectError',
'IntersectMimeType',
Expand All @@ -60,7 +60,7 @@
'ControlProvider': '.config',
'DataStoreConfig': '.config',
'DataStoreConfigMap': '.config',
'HierarchyConfig': '.config',
'IntersectConfig': '.config',
'ControlPlaneManager': '.control_plane.control_plane_manager',
'MessageCallback': '.control_plane.definitions',
'IntersectDataHandler': '.core_definitions',
Expand Down
90 changes: 27 additions & 63 deletions src/intersect_sdk_common/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,82 +3,30 @@
from dataclasses import dataclass, field
from typing import Annotated, Literal

from pydantic import BaseModel, ConfigDict, Field, PositiveInt
from pydantic import BaseModel, Field, PositiveInt

from .core_definitions import IntersectDataHandler

HIERARCHY_REGEX = r'^[a-z]((?!--)[a-z0-9-]){2,62}$'
HIERARCHY_REGEX = r'[a-z0-9][-a-z0-9]{2,62}'
"""
The hierarchy regex needs to be fairly restricted due to the number of different
systems we want to be compatible with. The rules:
The hierarchy regex should accomplish the following:

- Only allow unreserved characters (alphanumeric and .-~_): https://datatracker.ietf.org/doc/html/rfc3986#section-2.3
- Require lowercase letters to avoid incompatibilities with case-insensitive systems.
- MinIO has been found to forbid _ and ~ characters
- MinIO requires an alphanumeric character at the start of the string
- No adjacent non-alphanumeric characters allowed
- Range should be from 3-63 characters
- Be CLI friendly (don't start with hyphen) and URI friendly (don't use underscores)

The following commit tracks several issues with MINIO: https://code.ornl.gov/intersect/additive-manufacturing/ros-intersect-adapter/-/commit/fa71b791be0ccf1a5884910b5be3b5239cf9896f
This enables maximum integration with other systems (i.e. MINIO) without having to transform the hierarchy.
"""
# HIERARCHY_REGEX = r'^[a-z0-9]((?!--)[a-z0-9-]){2,62}$'
# If using this regex, you must configure the Pydantic ConfigDict regex_engine to be 'python-re' to support lookaheads
# Used to disallow multiple hyphens
# should work in HTML5 forms out of the gate

ControlProvider = Literal['mqtt5.0', 'amqp0.9.1']
"""The type of broker we connect to."""


class HierarchyConfig(BaseModel):
"""Configuration for registering this service in a system-of-system architecture."""

service: Annotated[str, Field(pattern=HIERARCHY_REGEX)]
"""
The name of this application - should be unique within an INTERSECT system
"""

subsystem: str | None = Field(default=None, pattern=HIERARCHY_REGEX)
"""
An associated subsystem / service-grouping of the system (should be unique within an INTERSECT system)
"""

system: Annotated[str, Field(pattern=HIERARCHY_REGEX)]
"""
Name of the "system", could also be thought of as a "device" (should be unique within a facility)
"""

facility: Annotated[str, Field(pattern=HIERARCHY_REGEX)]
"""
Name of the facility (an ORNL institutional designation, i.e. 'neutrons') (NOT abbreviated, should be unique within an organization)
"""

organization: Annotated[str, Field(pattern=HIERARCHY_REGEX)]
"""
Name of the organization (i.e. 'ornl') (NOT abbreviated) (should be unique in an INTERSECT cluster)
"""

def hierarchy_string(self, join_str: str = '') -> str:
"""Get the full hierarchy string. This is mostly used internally, but if you're developing a client, it could potentially be helpful.

Params
join_str: String used to separate different hierarchy parts in the full string (default: empty string).

Returns:
Single string, which will contain all system-of-system parts. For optional parts not configured (i.e. - no subsystem), they will be represented by a "-" character.
"""
if not self.subsystem:
return join_str.join([self.organization, self.facility, self.system, '-', self.service])
return join_str.join(
[
self.organization,
self.facility,
self.system,
self.subsystem,
self.service,
]
)

# we need to use the Python regex engine instead of the Rust regex engine here, because Rust's does not support lookaheads
model_config = ConfigDict(regex_engine='python-re')


@dataclass
class ControlPlaneConfig:
"""Configuration for interacting with a broker."""
Expand All @@ -89,6 +37,9 @@ class ControlPlaneConfig:
"""
# TODO - support more protocols and protocol versions as needed - see https://www.asyncapi.com/docs/reference/specification/v2.6.0#serverObject

system_name: Annotated[str, Field(pattern=HIERARCHY_REGEX)]
"""The highest level of namespacing on the broker address, provided by value configured by registry service administrator. Important for connecting different INTERSECT systems."""

username: Annotated[str, Field(min_length=1)]
"""
Username credentials for broker connection.
Expand Down Expand Up @@ -122,8 +73,7 @@ class ControlPlaneConfig:
NOTE: INTERSECT currently only supports AMQP and MQTT.
"""

# TODO default this to False once the registry service is in place
is_root: bool = True
is_root: bool = False
"""
Whether or not the broker credentials are for connecting as a root user.

Expand Down Expand Up @@ -181,3 +131,17 @@ def get_missing_data_store_types(self) -> set[IntersectDataHandler]:
if not self.minio:
missing.add(IntersectDataHandler.MINIO)
return missing


class IntersectConfig(BaseModel):
"""The response type used by INTERSECT-SDK Services and Clients to understand how to connect to the INTERSECT ecosystem.

This object should only be used by the SDK, and not users directly.
"""

service_name: str
"""The service namespacing of this microservice, reserved by user (or, if client, obtained from credential) on registry service."""
broker: ControlPlaneConfig
"""Control plane configuration the SDK should use with this Service."""
data_stores: Annotated[DataStoreConfigMap, Field(default_factory=lambda: DataStoreConfigMap())]
"""List of data plane configurations the SDK should use."""
4 changes: 2 additions & 2 deletions src/intersect_sdk_common/constants.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
"""These are miscellaneous constants used in INTERSECT which SDK users may obtain value from knowing about."""

SYSTEM_OF_SYSTEM_REGEX = r'^[a-z0-9][-a-z0-9.]*[-a-z0-9]$'
SYSTEM_OF_SYSTEM_REGEX = r'^[a-z0-9][-a-z0-9/]*[-a-z0-9]$'
"""
This is the regex used as a representation of a source/destination.
This is only needed externally if you are building a client, services can ignore this.

NOTE: for future compatibility reasons, we are NOT specifying the number of "parts" (separated by a '.') in this regex. All that matters is that you don't start or end with a period, or start with a hyphen.
NOTE: for future compatibility reasons, we are NOT specifying the number of "parts" (separated by a '/') in this regex. All that matters is that you don't start or end with a separator, or start with a hyphen.
"""

# see the internal schema file for full validation details
Expand Down
Loading
Loading