Skip to content

Add: airqo adaptor - #1757

Open
Dhie-boop wants to merge 6 commits into
OpenFn:mainfrom
Dhie-boop:add/new
Open

Dhie-boop wants to merge 6 commits into
OpenFn:mainfrom
Dhie-boop:add/new

Conversation

@Dhie-boop

Copy link
Copy Markdown

Summary

Add a new @openfn/language-airqo adaptor for the AirQo API, which provides air quality monitoring data from African cities.

Details

Functions added

Function Description
getRecentMeasurements(entityType, entityId, params) Fetch the latest readings for a site, device, grid, or cohort
getHistoricalMeasurements(entityType, entityId, params) Fetch past readings (up to 1 000 per call)
getAllHistoricalMeasurements(entityType, entityId, params) Fetch all past readings — auto-paginates and merges every page
listMetadata(resource, params) List all public sites, devices, grids, or cohorts
listSites(params) Convenience alias for listMetadata('sites')
listDevices(params) Convenience alias for listMetadata('devices')
listGrids(params) Convenience alias for listMetadata('grids')
listCohorts(params) Convenience alias for listMetadata('cohorts')
getGridSummary() Fetch every grid with its full list of nested site coordinates
getDailyForecast(params) Fetch a 3-day PM2.5 forecast for a device or site
getHourlyForecast(params) Fetch an hourly PM2.5 forecast for a device or site
toDhis2DataValues(measurements, mappingConfig, options) Convert AirQo measurements into DHIS2 dataValues shape for batch import

Other files added

  • src/Adaptor.js — all job-facing functions
  • src/Utils.jsrequest helper and prepareNextState
  • src/index.js — package entry point
  • test/Adaptor.test.js — 56 unit tests (Undici MockAgent, no live network calls)
  • configuration-schema.json — credential schema (token, optional baseUrl)
  • package.json — package definition
  • README.md — usage guide and DHIS2 workflow example

Key design decisions

  • Authentication is done via a token query parameter, as required by the AirQo API.
  • Input validation runs before every network call — invalid entity types, bad ISO dates, reversed date ranges, and missing tokens all throw clear, descriptive errors.
  • getAllHistoricalMeasurements handles pagination automatically. Callers get one flat array regardless of how many pages the API returns.
  • toDhis2DataValues produces a summary object alongside the dataValues array, listing exactly which measurements were skipped and why — useful for diagnosing org-unit mapping gaps without inspecting every record.
  • The token credential is never written to logs. The AirQo API takes auth as a query parameter, so the request logger redacts it before printing the request URL.

Testing

cd packages/airqo
pnpm install
pnpm test   
pnpm build  
Screenshot 2026-08-11 002052 Screenshot 2026-08-11 002301

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to
know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

Parts of this adaptor — including initial function scaffolding, JSDoc, unit test structure, and code review — were developed with assistance from Claude Opus (Anthropic) and GPT-5.5 (OpenAI). All AI-generated code was reviewed, tested, and validated by the author before inclusion. The final implementation, architectural decisions, and test coverage are the responsibility of the human author. No proprietary data or credentials were shared with any AI tool during development.

You can read more details in our
Responsible AI Policy

Review Checklist

Before merging, the reviewer should check the following items:

  • Does the PR do what it claims to do?
  • If this is a new adaptor, added the adaptor on marketing website ?
  • If this PR includes breaking changes, do we need to update any jobs in
    production? Is it safe to release?
  • Are there any unit tests? — Yes, 56 tests covering all functions and error paths.
  • Is there a changeset associated with this PR? Should there be? Note that
    dev only changes don't need a changeset. — Not added; this is a new adaptor with no existing consumers to break.
  • Have you ticked a box under AI Usage?

@Dhie-boop Dhie-boop changed the title Add: airqo adapto Add: airqo adaptor Aug 10, 2026
@Dhie-boop

Copy link
Copy Markdown
Author

@mtuchi @josephjclark @PiusKariuki
Opening this up for review. This adds a new airqo adaptor (AirQo air-quality API) with a bundled AirQo→DHIS2 sync workflow example in the README.

Happy to address any feedback.

@mtuchi

mtuchi commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Hey @Dhie-boop, thanks for putting this together! Nice to see AirQo getting adaptor support.

Could you say a bit more about the use case driving this? What jobs or integrations are you planning to build on top of it?

Asking because our usual guidance for a new adaptor is to start small with the generic HTTP helpers (get(), post(), request(), typically under an http namespace) and then grow resource-specific functions like getRecentMeasurements(entityType, entityId) as real usage shows which ones matter. It keeps the surface area we have to maintain and version small early on.

Would that work as a starting point here?

  1. Landing http.get() / http.post() / http.request() first, and
  2. Then promoting one or two of the highest-value helpers from this list, e.g. getRecentMeasurements(entityType, entityId), once we've seen how they get used in practice?

@Dhie-boop

Copy link
Copy Markdown
Author

Thanks for the thoughtful feedback! @mtuchi That approach makes sense to me.

Primarily I wanted to bridge the gap between environmental monitoring and national health/environmental reporting workflows. The goal is to create data pipelines that pull real-time air quality metrics (PM2.5 and PM10) from AirQo sensors and automatically sync them with respective platforms.

I completely agree with starting small
I will land the generic http.get(), http.post(), and http.request() functions under an http namespace, wrapping the existing Utils.js request() helper that already handles AirQo's token-based auth and the AirQo-specific error codes, so it just needs to be exposed generically instead of only powering the resource-specific functions.

Then, once we have our cron and webhook pipelines running end-to-end and can see the precise query parameters we rely on most, I will promote the highest-value helpers starting with getRecentMeasurements(entityType, entityId) and possibly getHistoricalMeasurements(entityType, entityId), since those already cover the sites/devices/grids/cohorts cases we need for reporting.

Let me know if you would like me to adjust the current PR to this scope.

@mtuchi

mtuchi commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

@Dhie-boop yes please update this pr to cover that

@Dhie-boop

Copy link
Copy Markdown
Author

@mtuchi Thanks will update.

Resolve conflict in pnpm-workspace.yaml: keep main's overrides (form-data ^2.5.6, json-sql>underscore, twilio>axios) and drop the local-only scriptShell override that had been accidentally committed.
@Dhie-boop

Copy link
Copy Markdown
Author

@mtuchi I made some changes checkout

I have remove the followings;

  • Removed resource-specific functions, retaining only getRecentMeasurements and getHistoricalMeasurements.
  • Introduced generic HTTP operations: get, post, and request, with convenience aliases under http.
  • Rewritten test suite for the adaptor, ensuring comprehensive coverage of the new API surface.
  • Updated documentation and changelog to reflect the reduced scope and new functionality.

All other resource-specific functions (and their now-unused helpers) from the
original draft have been removed. Resource-specific helpers can be promoted
individually later, once real usage patterns justify them.

Changes

  • src/Adaptor.js
    • Added get, post, and request as real, individually-documented
      top-level operations (each with full @public JSDoc), plus
      http = { get, post, request } as a plain convenience namespace alias.
    • Fixed a bug where request was re-exported directly from Utils.js
      (the raw infra helper, wrong signature/behavior for a job operation) —
      it's now a proper Adaptor-level operation with correct expandReferences
      handling and validation.
  • test/Adaptor.test.js — rewritten test suite (34 tests) covering
    getRecentMeasurements, getHistoricalMeasurements, and get/post/
    request (including their http.* aliases), using Undici MockAgent
    with no live network calls.
  • README.md — updated usage docs to match the reduced scope and the
    corrected get/post/request API surface.
  • CHANGELOG.md — updated initial-release notes to match the final
    scope.
  • ast.json — regenerated; now correctly lists get, post, request,
    getRecentMeasurements, and getHistoricalMeasurements as documented
    operations.

Validation

  • pnpm test — 34/34 passing
  • pnpm lint — clean
  • pnpm build — completes successfully (docs, types, and ast.json
    regenerated with no functional warnings)

@mtuchi mtuchi self-assigned this Aug 12, 2026
Comment thread packages/airqo/src/Adaptor.js Outdated

// Namespace alias grouping the generic helpers above; each member is already
// documented and exported individually as `get`, `post`, and `request`.
export const http = { get, post, request };

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dhie-boop please move the http functions in http.js
and then export it in index.js

export * as http from './http.js';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dhie-boop can you add a rectangle.png (512x190px) logo here ?

Comment thread packages/airqo/src/Adaptor.js Outdated
}
};

const ISO_8601_RE =

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do ? 🤔

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do ? 🤔

This ISO_8601_RE which I have rename it to ISO_8601_DATE_TIME_RE, is a regex which is used by assertMeasurementParams to validate startTime/endTime query params before they're compared as dates or sent to AirQo. so this rejects malformed strings and date-only strings without a timezone. It's actively used and covered by existing tests, so I thought keeping it.

Comment thread packages/airqo/src/Adaptor.js Outdated
* Also available as `http.get`.
*
* @example <caption>Get raw data from any AirQo path</caption>
* get('devices/measurements/sites/60d058c8048305120d######/recent');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be http.get()

Comment thread packages/airqo/src/Adaptor.js Outdated
* Also available as `http.post`.
*
* @example <caption>Post a body to any AirQo path</caption>
* post('devices/some-endpoint', { foo: 'bar' });

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

once moved to http.js this should be http.post()

Comment thread packages/airqo/src/Adaptor.js Outdated
* Also available as `http.request`.
*
* @example <caption>Make an arbitrary request</caption>
* request('GET', 'devices/metadata/grids');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http.request()

Comment thread packages/airqo/src/Utils.js Outdated
@@ -0,0 +1,84 @@
/**
* INVARIANT: Must export a function named `request`.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove the AI generated comments ?, unless they are very useful?

@mtuchi

mtuchi commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Hey @Dhie-boop i have reported couple of areas you can fix in this pr

@Dhie-boop

Dhie-boop commented Aug 12, 2026

Copy link
Copy Markdown
Author

Hey @Dhie-boop i have reported couple of fixes that you can address in this pr

Thanks for the valuable feedback @mtuchi
Am working on them right away.

@Dhie-boop

Copy link
Copy Markdown
Author

@mtuchi Have made the changes

  • Have extract get/post/request into src/http.js, exported as http namespace
    from index.js (job code now calls http.get/post/request)
  • Resize assets/rectangle.png to the required 512x190px
  • Trim stale/low-value comments left over from the http.js split
  • Split http.test.js out of Adaptor.test.js and give it a distinct mock
    origin (enableMockClient silently drops interceptors when two test
    files share the same origin)

Comment thread packages/airqo/src/Utils.js Outdated
const opts = {
parseAs: 'json',
errors,
baseUrl: baseUrl.endsWith('/') ? baseUrl : `${baseUrl}/`,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need "baseUrl.endsWith('/') ? baseUrl : ${baseUrl}/". Just baseUrl is enough

Comment thread packages/airqo/src/Adaptor.js Outdated
}
};

const assertAllowedValue = (value, allowedValues, label) => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this function to Utils.js, you can use it in adaptor functions util.assertAllowedValue

Comment thread packages/airqo/src/Adaptor.js Outdated

const VALID_ENTITY_TYPES = ['sites', 'devices', 'grids', 'cohorts'];

const assertNonEmptyString = (value, label) => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this to Utils.js

* - `'grids'` — a named geographic grid (e.g. a city or country)
* - `'cohorts'` — a user-defined group of devices
*
* @example <caption>Get recent measurements for a site</caption>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add an example

*
* @example <caption>Get recent measurements for a site</caption>

* @example <caption>Get recent measurements for a grid</caption>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add an example


* @example <caption>Get recent measurements for a grid</caption>

* @example <caption>Get recent measurements for a device</caption>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add an example

*
* @example <caption>Get historical measurements for a site</caption>

* @example <caption>Get historical measurements for a grid with date range</caption>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add an example

/**
* Make a generic authenticated GET request to any AirQo endpoint.
*
* @example <caption>Get raw data from any AirQo path</caption>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add an example

Comment thread packages/airqo/src/http.js Outdated
import { expandReferences } from '@openfn/language-common/util';
import * as util from './Utils.js';

const assertNonEmptyString = (value, label) => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please move this to Util.js

@Dhie-boop

Copy link
Copy Markdown
Author

Thanks @mtuchi for feedback.
Working on them.

Comment thread packages/airqo/src/Adaptor.js Outdated
*/

/**
* @typedef {Object} AirQoState

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This typedef is not used anywhere i would suggest you remove it. and run pnpm build to make sure docs are built okay @Dhie-boop

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I will do that

Comment thread packages/airqo/src/Utils.js Outdated
* @param {object} options
*/
export const request = (configuration = {}, method, path, options = {}) => {
const { baseUrl = 'https://api.airqo.net/api/v2', token } = configuration;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add support for X-Client-Secret https://platform.airqo.net/docs/api/getting-started/authentication/#additional-security-controls

The token by it self is not very secure since it's passed on query params

@mtuchi mtuchi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dhie-boop i have just couple of more change request for you

  • Remove the AirQoState and AirQoMeasurement typedef, since they are not used anywhere. I would i advice to add link to the airqo docs instead
  • For params docs especially in getMesurements functions please include link to airqo docs Eg: {@link https://platform.airqo.net/docs/api/for-partners/historical-data/#request-parameters Request parameters}
  • Add support for X-Client-Secrete in util.request. This encourages best security controls since token is being passed in query params
  • getRecentMeasurements() does not have request params from airqo api, please update the function signature
  • Add actual examples in each adaptor function.
* @example <caption>Get recent measurements for a site</caption>
* getRecentMeasurements("sites", "site123")

or in http.get()

* @example <caption>Get recent measurements for a site</caption>
* http.get("devices/measurements/sites/site123/recent")

Don't forget to update pr notes to reflect the latest changes, and run pnpm build before pushing changes

@Dhie-boop

Copy link
Copy Markdown
Author

@mtuchi summary of the new changes;

  • Simplified baseUrl handling in Utils.js
  • Moved assertAllowedValue and assertNonEmptyString into Utils.js.
  • Added optional clientSecret configuration support.
  • Added the X-Client-Secret header to AirQo requests.
  • Removed unused AirQoMeasurement and AirQoState typedefs.
  • Updated getRecentMeasurements to accept only entityType and entityId.
  • Added examples to AirQo adaptor and HTTP operation documentation.
  • Added links to the AirQo API documentation for request parameters.
  • Updated the configuration schema and README.
  • Updated unit tests for client-secret authentication and the revised API signature.
  • Regenerated the AirQo documentation and AST output.
  • Added a changeset for the AirQo patch release.

Output of the final pnpm build

Screenshot 2026-09-21 013029

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants