Skip to content

Add support for CREATE SECRET syntax (DuckDB) #1184

Closed

Description

@kspeeckaert

What are you building with SQLPage ?

SAP Message Analysis. Currently adding an interface that allows directly ingesting data from a blob storage (using the azure extension) and converting the XML files into DuckDB as a table (using the webbed extension).

What is your problem ?

I'm trying to store a connection string as a secret in DuckDB, which requires using the CREATE SECRET statement, which is not supported by the SQL parser in SQLPage.

Currently this results in the following error:

[2026-01-11T13:16:38.046Z ERROR sqlpage::webserver::database::execute_queries] Parsing failed: SQLPage couldn't understand the SQL file. Please check for syntax errors on line 13:
    CREATE OR REPLACE PERSISTENT SECRET $secret_name
                     ⬆️
    (
    Caused by:
        0: sql parser error: Expected: [EXTERNAL] TABLE or [MATERIALIZED] VIEW or FUNCTION after CREATE OR REPLACE, found: PERSISTENT at Line: 13, Column: 19
        1: import/upsert_container.sql contains a syntax error preventing SQLPage from parsing and preparing its SQL statements.

What are you currently doing ?

No workaround other than manually (through a SQL interface) running the SQL statement myself, as the extension I'm using (azure) does not support any other way to authenticate to the backend than using secrets (env. variables are deprecated).

Describe the solution you'd like

Support for the CREATE SECRET and DROP SECRET (documentation) statements.

Describe alternatives you've considered

None, as I don't see any alternatives at the moment (at least not through SQLPage).

Activity

  1. lovasoa commented on Jan 11, 2026

    @lovasoa
    Collaborator

    Hi! Can you test this on the latest builds (lovasoa/sqlpage:main on docker)? They use the duckdb dialect from sqlparser-rs when they detect a duckdb odbc connection.

  2. kspeeckaert commented on Jan 11, 2026

    @kspeeckaert
    Author

    Hi,
    I'm running SQLPage through homebrew on macOS. I can run the docker image through Podman, but I fear that trying to get the DuckDB ODBC driver installed on the Alpine image is not going to be easy (if at all possible), and I'm not that well versed in creating my own Docker images.

    I'd be more than happy to test it though, is there any other way to run the latest build?

  3. lovasoa commented on Jan 12, 2026

    @lovasoa
    Collaborator

    I'm adding a new duckdb docker image to simplify usage across platforms: #1187

  4. lovasoa commented on Jan 12, 2026

    @lovasoa
    Collaborator

    I published a pre release as lovasoa/sqlpage:main-duckdb

  5. kspeeckaert commented on Jan 13, 2026

    @kspeeckaert
    Author

    Thank you, I managed to get the container running and connecting to my database.

    The good news: there's no parsing error anymore on the CREATE SECRET syntax.

    The bad news: trying to run DML statements with parameters is a no-go. Both statements below fail.

    DROP PERSISTENT SECRET $secret_name IF EXISTS
    ;

    Error:

    �[2026-01-13T07:32:23.943Z ERROR sqlpage::webserver::database::execute_queries] Parsing failed: SQLPage couldn't understand the SQL file. Please check for syntax errors on line 13:
        -- Drop, as we might rename the secret
        DROP PERSISTENT SECRET $secret_name IF EXISTS
    .                                       ⬆️
        ;
        Caused by:
            0: sql parser error: Expected: identifier, found: $secret_name at Line: 13, Column: 24
            1: import/sa_upsert.sql contains a syntax error preventing SQLPage from parsing and preparing its SQL statements.
    
    CREATE OR REPLACE PERSISTENT SECRET $secret_name
    (
        TYPE AZURE
        ,CONNECTION_STRING $conn_string
        ,SCOPE ['AZURE://' || $sa_name
               ,'AZ://'    || $sa_name
               ,'ABFSS://' || $sa_name
               ,'ABFS://'  || $sa_name]
    );

    Error:

    �[2026-01-13T07:32:56.214Z ERROR sqlpage::webserver::database::execute_queries] Parsing failed: SQLPage couldn't understand the SQL file. Please check for syntax errors on lines 17 to 18:
            TYPE AZURE
            ,CONNECTION_STRING $conn_string
    .                                       ⬆️
            ,SCOPE ['AZURE://' || $sa_name
        
        Caused by:
            0: sql parser error: Expected: identifier, found: $conn_string at Line: 17, Column: 24
            1: import/sa_upsert.sql contains a syntax error preventing SQLPage from parsing and preparing its SQL statements.

    I've been thinking of alternatives, and so far the only thing I can think of is to write the statement to a filesqlpage_files table), and then execute it using sqlpage.run_sql(), as you described in #518.

  6. lovasoa commented on Jan 13, 2026

    @lovasoa
    Collaborator

    DML statements with parameters like DROP PERSISTENT SECRET $secret_name are not supported on duckdb's side, unfortunately.

    https://duckdb.org/docs/stable/configuration/secrets_manager

    Indeed, you'll need either sqlpage.exec or sqlpage.run_sql to do that. But be careful about permissions and escaping strings!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions