neo-client

neo-client is the Go client module for Machbase Neo.

It provides:

neo-client: the standard database/sql driver package
api: shared interfaces, options, and helper types
machnet: lower-level protocol and transport implementation used by neo-client

The examples in this module show how to connect to a Machbase Neo server, execute queries, append time-series records, and use the standard database/sql API.

Requirements

Go 1.22 or later
A reachable Machbase Neo server
A valid user account, such as sys / manager in a local development environment

Go Compatibility Guard

neo-client is consumed as a database/sql/driver implementation by downstream applications, so it must remain buildable with the Go version declared in go.mod. Run the compatibility check before release or when touching client code:

./scripts/test-go-compat.sh

For fixed-cardinality numeric ARRAY values, sparse values, and append element projection, see ARRAY type and sparse append.

The script builds all packages and test packages outside the workspace (GOWORK=off) with the Go toolchain declared in go.mod, which catches newer language features and standard library APIs that a downstream user on the declared Go version could not build.

The examples in this repository use the native TCP endpoint, usually 127.0.0.1:5656.

Install

go get github.com/machbase/neo-client/v2

Package Layout

`client`

This package provides a standard Go database/sql driver on top of the native TCP client. Use it when you want to integrate Machbase Neo with libraries or application code that already expect the standard database/sql interfaces.

`api`

This package contains a machbase specific data type and type definitions.

`machnet`

This is the lower-level client implementation used internally by machgo. Most application code should not need to import it directly.

Quick Start

Connect and Query

The following example uses the standard database/sql package and reads from the M$SYS_TABLES system table.

package main

import (
	"context"
	"database/sql"
	"fmt"

	_ "github.com/machbase/neo-client/v2"
)

func main() {
	db, err := sql.Open("machbase", "server=tcp://sys:manager@127.0.0.1:5656")
	if err != nil {
		panic(err)
	}
	defer db.Close()

	ctx := context.Background()
	rows, err := db.QueryContext(ctx, `SELECT NAME, ID, TYPE FROM M$SYS_TABLES ORDER BY NAME`)
	if err != nil {
		panic(err)
	}
	defer rows.Close()

	for rows.Next() {
		var (
			name string
			id   int64
			typ  int
		)
		if err := rows.Scan(&name, &id, &typ); err != nil {
			panic(err)
		}
		fmt.Println(name, id, typ)
	}

	if err := rows.Err(); err != nil {
		panic(err)
	}
}

Create Table and Insert Rows

The following example uses database/sql to create a tag table and insert records with ExecContext.

CREATE TAG TABLE IF NOT EXISTS example (
    name VARCHAR(100) PRIMARY KEY,
	time DATETIME BASE TIME,
    value DOUBLE
);

Then insert rows:

package main

import (
	"context"
	"database/sql"
	"fmt"
	"time"

	_ "github.com/machbase/neo-client/v2"
)

func main() {
	dsn := "server=tcp://sys:manager@127.0.0.1:5656"

	db, err := sql.Open("machbase", dsn)
	if err != nil {
		panic(err)
	}
	defer db.Close()

	ctx := context.Background()

	_, err = db.ExecContext(ctx, `CREATE TAG TABLE IF NOT EXISTS EXAMPLE (
		NAME   VARCHAR(100)  PRIMARY KEY,
		TIME   DATETIME      BASE TIME,
		VALUE  DOUBLE
	)`)
	if err != nil {
		panic(err)
	}

	ts := time.Now()
	for i := 0; i < 10; i++ {
		rec := []any{
			"example-client",
			ts.Add(time.Duration(i)*time.Second),
			3.14*float64(i),
		}
		result, err := db.ExecContext(ctx, `INSERT INTO EXAMPLE VALUES (?, ?, ?)`, rec...)
		if err != nil {
			panic(err)
		}
		affected, err := result.RowsAffected()
		if err != nil {
			panic(err)
		}
		fmt.Println("Rows affected:", affected)
	}
}

Transactions

Machbase supports BEGIN / COMMIT / ROLLBACK on regular tables created with CREATE TABLE (transaction tables). TAG/LOG tables do not support transactions; DML on a TAG/LOG table inside a transaction fails with MACHCLI-ERR-2362.

The standard database/sql transaction API works as expected:

tx, err := db.BeginTx(ctx, nil)
if err != nil {
	panic(err)
}
if _, err := tx.ExecContext(ctx, `INSERT INTO EXAMPLE_TX VALUES (?, ?, ?)`, name, ts, value); err != nil {
	tx.Rollback()
	panic(err)
}
if err := tx.Commit(); err != nil {
	panic(err)
}

For less boilerplate, the package provides closure helpers that commit when the function returns nil, roll back when it returns an error, and roll back then re-panic when it panics:

import client "github.com/machbase/neo-client/v2"

err := client.Tx(ctx, db, func(tx *sql.Tx) error {
	if _, err := tx.ExecContext(ctx, `INSERT INTO EXAMPLE_TX VALUES (?, ?, ?)`, name, ts, value); err != nil {
		return err // automatic ROLLBACK
	}
	return nil // automatic COMMIT
})

// TxConn runs the transaction on a specific connection acquired via db.Conn(ctx).
conn, _ := db.Conn(ctx)
defer conn.Close()
err = client.TxConn(ctx, conn, func(tx *sql.Tx) error {
	// ...
	return nil
})

The error returned by the closure is returned as-is, so errors.Is / errors.As keep working; returning a sentinel error is the idiomatic way to force a rollback. Transaction options (isolation level, read-only) are not supported by machbase and are rejected by the driver. See transaction.go for a complete runnable example and docs/transaction-helper.md for details.

Append Rows with Appender API

For high-throughput time-series ingestion, use the appender API instead of row-by-row INSERT. The appender buffers records on the client and streams them to the server over a dedicated channel, which is orders of magnitude faster than individual INSERT statements for bulk loads.

import client "github.com/machbase/neo-client/v2"

appender := &client.Appender{}
// Connect to a subset of columns: Append() below sends only these three
// values per row, and every other column of the table is written as NULL.
if err := appender.Connect(ctx, dsn, "EXAMPLE", "NAME", "TIME", "VALUE"); err != nil {
	panic(err)
}
defer func() {
	successCount, failCount, err := appender.Close() // flushes remaining buffered rows
	if err != nil {
		panic(err)
	}
	fmt.Println("Append finished. Success:", successCount, "Fail:", failCount)
}()

for _, rec := range records {
	// Exactly one value per column passed to Connect, in the same order.
	if err := appender.Append(rec.Name, rec.Time, rec.Value); err != nil {
		panic(err)
	}
}

Key points:

Column selection: the column list passed to Connect (or WithInputColumns) defines exactly which columns each Append call provides, in order. Columns of the table that are not listed are written as NULL. If the table above had additional columns beyond NAME, TIME, VALUE, they would all receive NULL.
Omitting the column list (e.g. appender.Connect(ctx, dsn, "EXAMPLE")) means the appender targets every column of the table: each Append call must then supply a value for every column — nil included — or it fails with a value-count error.
Append buffers rows; call Flush() to force a send, or rely on Close() to flush and report per-session success/fail counts.
Buffering behavior can be tuned with WithBatchMaxRows, WithBatchMaxBytes, and WithBatchMaxDelay.
The appender works with tag, log, and transaction tables, but it bypasses SQL — appends are not part of any transaction.

See append.go for a complete runnable example and ARRAY type and sparse append for appending ARRAY columns.

Scan Rows into Structs

Instead of listing every destination in column order, you can map columns to struct fields with the db tag. The helpers accept the *sql.Rows you already have, so they work with the standard database/sql API.

See the SQL convenience API reference for mapping rules, options, streaming, error handling, and named arguments.

import client "github.com/machbase/neo-client/v2"

type TagRecord struct {
	Name  string    `db:"NAME"`
	Time  time.Time `db:"TIME"`
	Value float64  `db:"VALUE"`

	cached string // unexported and untagged fields are ignored
}

records, err := client.Select[TagRecord](ctx, db,
	`SELECT NAME, TIME, VALUE FROM EXAMPLE WHERE NAME = ? ORDER BY TIME LIMIT 100`, "sensor-1")

Available helpers:

Function	Purpose
`Select[T](ctx, q, query, args...)`	Run a query and scan every row into `[]T`
`Get[T](ctx, q, query, args...)`	Run a query and scan the first row; returns `sql.ErrNoRows` when empty
`ScanAll[T](rows)` / `ScanOne[T](rows)`	Same, for rows the caller already opened
`ScanEach[T](rows, fn)`	Stream rows one at a time, keeping memory constant
`NewCursor[T](rows)`	Explicit `Next` / `Value` / `Err` iterator
`ScanStruct(rows, &dest)`	Scan the current row without calling `rows.Next()`
`ScanRow(rows, &dest)` / `ScanRows(rows, &slice)`	Non-generic forms

T may be a struct, a pointer to a struct, a scalar for single-column queries, or map[string]any.

Mapping rules:

The tag key is db, and json is used as a fallback so existing DTOs work unchanged.
Column names are matched case-insensitively, so db:"id" matches an ID column.
db:"-" excludes a field, and an untagged field is excluded as well. Call WithNameMapper(client.NameMapperIdentity()) to map untagged fields by name.
Embedded structs are flattened; a named nested struct is addressed as parent.child.
A NULL column can be received either as a *T field, which becomes nil, or as sql.Null[T].

By default the mapping is strict: a column with no matching field and a field with no matching column are both errors, which keeps a changed SELECT * from silently dropping values. Relax it per call with WithLaxColumns() or WithLaxFields().

A DATETIME column scanned into a string, int64, or time.Time field honors extra db tag options, named to match the machbase-neo HTTP API's timeformat/tz query parameters:

type Row struct {
	Time  string    `db:"TIME,timeformat=2006-01-02 15:04:05,tz=Local"` // custom layout + display zone
	Epoch int64     `db:"TIME,timeformat=ms"`                          // epoch in ms
	At    time.Time `db:"TIME,tz=UTC"`                                 // per-field zone override
}

timeformat=<Go time layout> — for string/*string fields, a Go time layout (or ns/us/ms/s to render the epoch as a numeric string).
timeformat=ns|us|ms|s — for int64/*int64 fields, the epoch unit.
tz=<IANA name>|Local|UTC — for string/time.Time fields (and their pointer forms).

These options apply even without a tag: any string, int64, or time.Time field matched to a DATETIME column uses WithDateTime(timeformat, tz) as its default, or timeformat="2006-01-02 15:04:05.999" and tz="Local" when WithDateTime isn't set either. A field's own tag always takes precedence over WithDateTime. Columns that aren't actually DATETIME (e.g. a VARCHAR into a string field) are unaffected and scan normally — the ambiguity is resolved at scan time by checking whether the value is actually a time.Time, not by the field's Go type. The same applies to sql.NullTime/sql.Null[time.Time]/sql.Null[int64]/sql.NullString/sql.Null[string], and to scalar (non-struct) targets like Select[time.Time] or Select[string] — since a bare scalar has no tag, only WithDateTime/the built-in default can customize it.

Select, ScanAll and ScanRows materialize the whole result set, so they stop with ErrScanTooManyRows beyond WithMaxRows, which defaults to 1000. Raise it with WithMaxRows(n), remove it with WithMaxRows(0), or stream the query with ScanEach or NewCursor, which have no limit.

rows, err := db.QueryContext(ctx, `SELECT NAME, TIME, VALUE FROM EXAMPLE`)
if err != nil {
	panic(err)
}
defer rows.Close() // the helpers never close rows they receive

var total float64
err = client.ScanEach(rows, func(rec TagRecord) error {
		total += rec.Value
	return nil
})

Build Named Parameters from a Struct or Map

NamedArgs turns a struct or a map[string]any into sql.Named arguments using the same db tags. It never inspects or rewrites the SQL text: the server resolves the :name placeholders itself.

type condition struct {
	Name string    `db:"name"`
	From time.Time `db:"from"`
	To   time.Time `db:"to"`
}

args, err := client.NamedArgs(condition{Name: "sensor-1", From: begin, To: end})
if err != nil {
	panic(err)
}

records, err := client.Select[TagRecord](ctx, db, `
	SELECT NAME, TIME, VALUE FROM EXAMPLE
	 WHERE NAME = :name AND TIME BETWEEN :from AND :to`, args...)

Named parameters require a server that reports parameter-name metadata (Machbase v8.7.0 or later). Check it with client.SupportsNamedParameters(ctx, db); otherwise the query fails with client.ErrNamedParamsUnsupported and positional ? markers must be used.

Use the Standard `database/sql` Driver

If your application already uses database/sql, import github.com/machbase/neo-client for driver registration and connect with sql.Open.

package main

import (
	"context"
	"database/sql"
	"fmt"

	_ "github.com/machbase/neo-client/v2"
)

func main() {
	dsn := "server=tcp://sys:manager@127.0.0.1:5656;fetch_rows=777;statement_cache=off;io_metrics=true"

	db, err := sql.Open("machbase", dsn)
	if err != nil {
		panic(err)
	}
	defer db.Close()

	ctx := context.Background()
	rows, err := db.QueryContext(ctx, `SELECT * FROM M$SYS_TABLES ORDER BY NAME`)
	if err != nil {
		panic(err)
	}
	defer rows.Close()

	cols, err := rows.Columns()
	if err != nil {
		panic(err)
	}
	fmt.Println("Columns:", cols)
	for rows.Next() {
		// scan values here
	}
	if err := rows.Err(); err != nil {
		panic(err)
	}
}

Supported DSN forms for the standard driver:

Server value only:
- host
- host:port
URL form such as tcp://user:password@host:port/database?as=proxy&fetch_rows=100
Key-value pairs separated by ;:
- key=value;key=value;...
- example: user=sys;password=manager;server=127.0.0.1:5656

For key-value DSN syntax:

Value may be quoted with "..." or '...'.
; inside quoted values is treated as a literal character.
Escapes are supported inside quoted values with backslash:
- \" for " in double-quoted values
- \' for ' in single-quoted values
- \\ for \
Unterminated or mismatched quotes return a parse error.

Examples:

user="sys as demo";password="12;34";server=127.0.0.1:5656;
user='sys as demo';password='12;34';server=127.0.0.1:5656;
password="a\\\"b";server=127.0.0.1:5656;

Supported DSN keys include:

server: server address such as tcp://sys:manager@127.0.0.1:5656
host, port: explicit server fields (default port=5656)
user, password : login user and password
database: database name
timezone: timezone for datetime values; accepts an IANA location such as America/New_York or an offset such as +09:00 or -0530 (default: UTC)
auth_mode: authentication mode (password or challenge)
fetch_rows, fetchrows: fetch batch size (default: 1000)
statement_cache, statementcache: auto, on, or off (default: auto)
io_metrics, iometrics: true or false
alternative_servers: one or more comma-separated server addresses such as 127.0.0.2:5656,backup.example.com:5657

When auth_key_file or auth_key_pem is set and auth_mode is omitted, the driver treats it as challenge authentication implicitly.

auth_key_file: private key file path for auth_mode=challenge
auth_key_pem: inline private key PEM content for auth_mode=challenge
auth_sig_scheme: challenge authentication signature scheme.

URL query parameters use the same option names. For example:

tcp://sys:manager@127.0.0.1:5656/DATABASE_A?statement_cache=on&io_metrics=true

Unknown keys are errors in key-value DSNs but are ignored in URL query strings.

The URL path also selects the initial database, for example tcp://sys:manager@127.0.0.1:5656/DATABASE_A. The driver selects the configured database on every new physical connection. If application code executes USE directly, the driver restores the configured database before that connection is reused from the pool.

The standard driver follows database/sql pooling through sql.DB. On servers that support transaction tables, Begin, BeginTx, Commit, and Rollback execute explicit transactions. Only the default isolation level is accepted; read-only and custom isolation options return an error. LastInsertId is not supported.

Machbase 8.6.0 DECIMAL and Named Parameters

Machbase 8.6.0 provides exact DECIMAL values, nullable column information, and named parameters:

import "database/sql"
import client "github.com/machbase/neo-client/v2"

amount, err := client.ParseDecimal("1234567890.125", 30, 3)
if err != nil {
	panic(err)
}
result, err := conn.ExecContext(ctx,
	"INSERT INTO payments(id, amount) VALUES (:id, :amount)",
	sql.Named("id", int32(1)),
	sql.Named("amount", amount),
)
if err != nil {
	panic(err)
}

The database/sql driver accepts sql.Named and returns DECIMAL query values as exact strings. Parameter names are matched case-insensitively, a repeated marker reuses one supplied value, and named and positional arguments cannot be mixed. client.NamedArgs builds the sql.Named list from a struct or a map.

When connected to Machbase 8.5.x, use positional ? parameters with the table and data types supported by that server version. Named parameters and Machbase 8.6.0 data types are not available, and nullable column information may be unknown (ColumnType.Nullable() returns ok=false).

See Machbase 8.6.0 Go client guide for Transaction table edition limits, DECIMAL/NULL scan patterns, transaction and appender usage, prepared statements, and Machbase 8.5.x compatibility.

Running the Included Examples

Runnable examples are included under _example/.

Run the query example:

go run ./_example/query.go -s 127.0.0.1:5656 -u sys -p manager

Run the append example:

go run ./_example/append.go -s 127.0.0.1:5656 -u sys -p manager

Run the insert example:

go run ./_example/insert.go -s 127.0.0.1:5656 -u sys -p manager

Run the struct-tag scan and named parameter example:

go run ./_example/scanbytag.go -s 127.0.0.1:5656 -u sys -p manager

Common DSN Options

server=tcp://user:password@host:port: full server URL
database=DB_NAME (or URL path): initial database for each physical connection
timezone=America/New_York (or tz=+09:00): timezone for datetime values; defaults to UTC
fetch_rows=777: override fetch batch size
statement_cache=auto|on|off: control statement reuse; defaults to auto
io_metrics=true|false: enable or disable I/O metrics
alternative_servers=host1:port1,host2:port2: one or more alternative server addresses, tried in order

Notes

Always close Rows, Stmt, sql.Conn, and sql.DB objects after use. The struct scan helpers never close the rows they receive.
Appender.Close() returns success and failure counts for the append session.
On servers that support transaction tables, the standard driver supports explicit transactions. LastInsertId remains unsupported.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
_example		_example
api		api
docs		docs
machnet		machnet
scripts		scripts
.gitignore		.gitignore
README.md		README.md
appender.go		appender.go
appender_test.go		appender_test.go
array_buffer_test.go		array_buffer_test.go
config.go		config.go
config_test.go		config_test.go
driver.go		driver.go
driver_test.go		driver_test.go
go.mod		go.mod
named.go		named.go
scan.go		scan.go
scan_test.go		scan_test.go
scanstruct.go		scanstruct.go
structmap.go		structmap.go
tx.go		tx.go
types.go		types.go
types_test.go		types_test.go
util.go		util.go
util_test.go		util_test.go

Navigation Menu

Repository navigation

About

Repository files navigation

neo-client

Requirements

Go Compatibility Guard

Install

Package Layout

`client`

`api`

`machnet`

Quick Start

Connect and Query

Create Table and Insert Rows

Transactions

Append Rows with Appender API

Scan Rows into Structs

Build Named Parameters from a Struct or Map

Use the Standard `database/sql` Driver

Machbase 8.6.0 DECIMAL and Named Parameters

Running the Included Examples

Common DSN Options

Notes

See Also

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

neo-client

Requirements

Go Compatibility Guard

Install

Package Layout

client

api

machnet

Quick Start

Connect and Query

Create Table and Insert Rows

Transactions

Append Rows with Appender API

Scan Rows into Structs

Build Named Parameters from a Struct or Map

Use the Standard database/sql Driver

Machbase 8.6.0 DECIMAL and Named Parameters

Running the Included Examples

Common DSN Options

Notes

See Also

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

`client`

`api`

`machnet`

Use the Standard `database/sql` Driver