Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 46 additions & 2 deletions .github/workflows/desktop-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,43 @@ jobs:
dist/filecollector-windows-*-x64.zip.sha256
if-no-files-found: error

# 可选的 LibreOffice 转换器包 (Office→PDF 视觉转换链路): 主包体积不变,
# 用户按需下载解压到便携包根目录。LO_VERSION 固定于脚本内, 升级时手动更新。
# PR 不构建 (350MB 下载纯耗时); main/tag 上构建, tag 时随 Release 发布。
converter-package:
name: Build Windows LibreOffice converter package
runs-on: windows-latest
if: github.event_name != 'pull_request'
defaults:
run:
shell: msys2 {0}
steps:
- name: Check out source
uses: actions/checkout@v4

- name: Set up MSYS2 UCRT64
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: false
install: >-
unzip
zip

- name: Build LibreOffice converter package
run: |
chmod +x tools/build-converter-package.sh
tools/build-converter-package.sh

- name: Upload converter package
uses: actions/upload-artifact@v4
with:
name: filecollector-windows-converter-x64
path: |
dist/filecollector-converter-windows-*-x64.zip
dist/filecollector-converter-windows-*-x64.zip.sha256
if-no-files-found: error

macos:
name: Build macOS app bundle (ARM64)
runs-on: macos-latest
Expand All @@ -188,7 +225,7 @@ jobs:

publish:
name: Publish desktop packages
needs: [lint, deb, flatpak, windows, macos]
needs: [lint, deb, flatpak, windows, macos, converter-package]
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-24.04
permissions:
Expand Down Expand Up @@ -216,6 +253,12 @@ jobs:
name: filecollector-windows-x64
path: release-assets

- name: Download Windows converter package
uses: actions/download-artifact@v4
with:
name: filecollector-windows-converter-x64
path: release-assets

- name: Download macOS package
uses: actions/download-artifact@v4
with:
Expand All @@ -228,8 +271,9 @@ jobs:
test "$(find release-assets -maxdepth 1 -name '*.deb' -type f | wc -l)" -eq 1
test "$(find release-assets -maxdepth 1 -name '*.flatpak' -type f | wc -l)" -eq 1
test "$(find release-assets -maxdepth 1 -name 'filecollector-windows-*-x64.zip' -type f | wc -l)" -eq 1
test "$(find release-assets -maxdepth 1 -name 'filecollector-converter-windows-*-x64.zip' -type f | wc -l)" -eq 1
test "$(find release-assets -maxdepth 1 -name 'filecollector-macos-*-arm64.zip' -type f | wc -l)" -eq 1
test "$(find release-assets -maxdepth 1 -name '*.sha256' -type f | wc -l)" -eq 3
test "$(find release-assets -maxdepth 1 -name '*.sha256' -type f | wc -l)" -eq 4

- name: Create or update GitHub Release
uses: softprops/action-gh-release@v2
Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,14 @@ FileCollector 支持自动将二进制文件转换为 Markdown 格式,无需
- **文档文件**(PDF、DOCX、PPTX、XLSX、ODT、ODP、ODS、RTF 等):先通过 LibreOffice 转换为 PDF,再通过 `pdftoppm` 渲染为图片序列,逐页发送给 VLM。
- **转换缓存**:转换结果缓存在工作目录下的 `.filecollector_cache/` 目录中,基于文件 SHA256 哈希判断是否需要重新转换,避免重复处理。
- **可配置扩展名**:在 AI 设置对话框中可自定义允许被 VLM 处理的二进制文件扩展名列表,修改后自动重新评估预处理队列。
- **失败原因透明**:转换失败时队列行与预览区会显示具体原因(如缺少 LibreOffice),不再只提示"转换失败"。

### 处理方式:视觉转换 / 文本抽取

在 **AI 设置 → VLM** 的 **处理方式** 下拉中可选两种二进制文档处理流程:

- **视觉转换**(默认):文档 → PDF → 页面图片 → VLM/OCR,保留图表、公式、表格等视觉结构。Office 文档需要 LibreOffice —— Windows 用户可下载独立的**转换器包**(Release 附件 `filecollector-converter-windows-x64-*.zip`),解压到便携包根目录(与 `bin/` 同级)即可,应用会自动探测 `tools/LibreOffice/program/soffice.exe`;已安装官方 LibreOffice 的用户无需转换器包,应用同样自动探测。
- **文本抽取**:纯本地解析 DOCX/XLSX/PPTX/ODT/ODP/ODS(ZIP+XML 结构)与 PDF(内置 `pdftotext`),**无需联网、无需 AI 凭据**,秒级完成;代价是丢失图表、公式与嵌入图片,仅适合纯文字型文档。扫描版 PDF 抽不出文本时自动回退视觉转换流程。两种模式的结果在缓存中分别标记,互不污染,切换模式后对已有条目点"重新转换"即可按新模式重跑。

### VLM 服务商

Expand Down
8 changes: 8 additions & 0 deletions README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,14 @@ FileCollector can automatically convert binary files into Markdown format, elimi
- **Document files** (PDF, DOCX, PPTX, XLSX, ODT, ODP, ODS, RTF, etc.): First converted to PDF via LibreOffice, then rendered as image sequences via `pdftoppm`, and sent page-by-page to VLM.
- **Conversion cache**: Converted results are cached in the `.filecollector_cache/` directory under the working directory. The system uses SHA256 file hashes to determine whether re-conversion is needed, avoiding redundant processing.
- **Configurable extensions**: In the AI Settings dialog, you can customize the list of binary file extensions allowed for VLM processing. Changes automatically trigger re-evaluation of the preprocessing queue.
- **Transparent failure reasons**: When conversion fails, the queue row and preview area show the concrete reason (e.g., missing LibreOffice) instead of a bare "conversion failed".

### Processing mode: Visual conversion / Text extraction

Choose how binary documents are processed via the **Processing Mode** dropdown in **AI Settings → VLM**:

- **Visual conversion** (default): document → PDF → page images → VLM/OCR, preserving charts, formulas, and table layouts. Office documents need LibreOffice — Windows users can download the standalone **converter package** (Release asset `filecollector-converter-windows-x64-*.zip`) and extract it into the portable package root (alongside `bin/`); the app auto-detects `tools/LibreOffice/program/soffice.exe`. If the official LibreOffice is already installed, no converter package is needed — it is auto-detected as well.
- **Text extraction**: Pure local parsing of DOCX/XLSX/PPTX/ODT/ODP/ODS (ZIP+XML) and PDF (built-in `pdftotext`) — **no network, no AI credentials required**, finishes in seconds; the trade-off is losing charts, formulas, and embedded images, so it suits text-only documents. Scanned PDFs that yield no text automatically fall back to the visual conversion flow. Results of both modes are tagged separately in the cache; after switching modes, click "retry" on existing items to re-run with the new mode.

### VLM Providers

Expand Down
2 changes: 2 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@

「设置 → AI 设置」中的 **AI 助手(侧边栏)** 与 **视觉语言模型(VLM)** 均支持**多模型配置方案(Model Profile)**:可同时预置多套模型配置,各自通过下拉菜单一键切换。

VLM 顶部还有 **处理方式** 下拉,决定二进制文档走哪条管线:**视觉转换**(默认,文档渲染为图片后送 VLM/OCR,保留图表/公式/表格;Office 文件需要 LibreOffice,Windows 可下载 Release 里的转换器包解压到便携包根目录)或 **文本抽取**(本地直接解析 Office/PDF,快速离线且无需 AI 凭据,但丢失图表/公式/嵌入图片;扫描版 PDF 自动回退视觉转换)。两种模式的缓存结果分开标记,切换后对已有条目点"重新转换"即可按新模式重跑。

**配置文件位置**:`~/.config/filecollector/settings.json`

#### 配置文件格式
Expand Down
2 changes: 2 additions & 0 deletions docs/USAGE_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ After selecting a working directory, use the search box at the top of the left-s

Both **AI Assistant (Sidebar)** and **Vision-Language Model (VLM)** in "Settings → AI Settings" support **Model Profiles**: you can pre-configure multiple model setups and switch between them instantly from a dropdown menu.

The VLM section also has a **Processing Mode** dropdown that selects the pipeline for binary documents: **Visual conversion** (default — documents are rendered to images and sent to VLM/OCR, keeping charts/formulas/tables; Office files need LibreOffice — Windows users can download the converter package from Release assets and extract it into the portable package root) or **Text extraction** (parses Office/PDF locally — fast, offline, no AI credentials required, but charts/formulas/embedded images are lost; scanned PDFs automatically fall back to visual conversion). Cached results are tagged per mode; after switching, click "retry" on existing items to re-run with the new mode.

**Config file location**: `~/.config/filecollector/settings.json`

#### Config file format
Expand Down
27 changes: 26 additions & 1 deletion docs/building/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,13 @@ chmod +x tools/build-windows-package.sh
tools/build-windows-package.sh
```

脚本会下载并静态编译 cmark-gfm,运行 Meson 单元测试,递归收集 UCRT64 运行时 DLL(含 GTK 图像加载器的间接依赖)、GTK 动态模块、GSettings schema、完整 Adwaita/hicolor 图标主题,以及项目实际使用的两个 XApp symbolic 图标。然后它会验证所有源码引用的 symbolic 图标都能在包内主题中解析,并在打包前后用 `tools/verify_windows_package.py` 对全部 PE 文件做导入表闭包校验;运行时 DLL 数量不足或存在未解析的非系统 DLL 时,构建直接失败。
脚本会下载并静态编译 cmark-gfm,运行 Meson 单元测试,递归收集 UCRT64 运行时 DLL(含 GTK 图像加载器的间接依赖)、GTK 动态模块、GSettings schema、完整 Adwaita/hicolor 图标主题、zh_CN 翻译文件(.mo)、项目实际使用的两个 XApp symbolic 图标,以及 ZIP 导出依赖的 zip.exe(含 msys-2.0.dll)。然后它会验证所有源码引用的 symbolic 图标都能在包内主题中解析,并在打包前后用 `tools/verify_windows_package.py` 对全部 PE 文件做导入表闭包校验;运行时 DLL 数量不足或存在未解析的非系统 DLL 时,构建直接失败。

> 注意:locale 与 zip.exe 曾因 MSYS2 参数路径转换(`meson --prefix=/usr` 的
> 参数被改写为 MSYS2 安装根下的路径)而静默缺失,导致 v4.9.1 及之前的
> Windows 包为纯英文界面且 ZIP 导出不可用。脚本现通过 `MSYS2_ARG_CONV_EXCL`
> 排除转换,并在打包后硬性校验 `locale/zh_CN/LC_MESSAGES/filecollector.mo`
> 与 `bin/zip.exe` 存在于 ZIP 内。

排错提示:DLL 收集脚本按 `MSYSTEM_PREFIX` 判定运行时目录,并用 `ldd` 递归解析依赖。若本地构建得到的 `bin/` 里几乎没有 DLL,先确认是在 MSYS2 **UCRT64** 终端中执行(`echo $MSYSTEM_PREFIX` 应为 `/ucrt64`),以及 `ldd staging/bin/filecollector.exe` 能正常解析。

Expand All @@ -45,6 +51,25 @@ dist/filecollector-windows-<meson.build 中的版本>-x64.zip
dist/filecollector-windows-<meson.build 中的版本>-x64.zip.sha256
```

## LibreOffice 转换器包(可选)

主包不捆绑 LibreOffice(体积原因)。Office 文档的**视觉转换**链路(soffice → PDF → pdftoppm → VLM)依赖 LibreOffice,通过独立的转换器包交付:

```bash
tools/build-converter-package.sh
```

脚本下载官方 LibreOffice MSI(版本固定于脚本内 `LO_VERSION`),用 `msiexec /a` 管理员解包(不写注册表、不进 Program Files),打包为:

```text
dist/filecollector-converter-windows-<LO_VERSION>-x64.zip
dist/filecollector-converter-windows-<LO_VERSION>-x64.zip.sha256
```

布局为 `tools/LibreOffice/program/soffice.exe`,解压到便携包根目录(与 `bin/` 同级)即生效;应用通过 `BinaryConverter.resolve_soffice()` 自动探测。CI 中该包在 main/tag 推送时构建,tag 推送时随 Release 发布。

注意:`msiexec` 的 `/a` 参数从 bash 直接传入会被 MSYS2 改写为路径(与 `--prefix=/usr` 同类陷阱),脚本通过 `cmd //c` 复合字符串绕过。

## 试运行

将 ZIP 解压后,始终通过下列启动器启动:
Expand Down
3 changes: 3 additions & 0 deletions meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ sources = files(
'src/services/binary_converter.vala',
'src/services/token_estimator.vala',
'src/services/binary_preprocessor.vala',
'src/services/office_text_extractor.vala',
'src/services/config_manager.vala',
'src/services/file_content_reader.vala',
'src/services/file_generator.vala',
Expand Down Expand Up @@ -163,8 +164,10 @@ sources = files(
)

# win32_dpapi_shim.c 仅 Windows 构建需要 (DPAPI 封装), 其它平台不存在该文件。
# win32_console_shim.c 仅 Windows 构建需要 (GUI 子系统下的 CLI 控制台接管)。
if host_machine.system() == 'windows'
sources += files('src/win32_dpapi_shim.c')
sources += files('src/win32_console_shim.c')
endif

# macos_keychain_shim.c 仅 macOS 构建需要 (SecItem* 封装), 其它平台不存在该文件。
Expand Down
1 change: 1 addition & 0 deletions po/POTFILES
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ src/services/ai_client.vala
src/services/ai_types.vala
src/services/binary_converter.vala
src/services/binary_preprocessor.vala
src/services/office_text_extractor.vala
src/services/config_manager.vala
src/services/file_content_reader.vala
src/services/file_generator.vala
Expand Down
132 changes: 132 additions & 0 deletions po/zh_CN.po
Original file line number Diff line number Diff line change
Expand Up @@ -2693,3 +2693,135 @@ msgstr "正在预处理 0/0 个文件..."
#: src/widgets/vlm_queue_controller.vala:146
msgid "Cancel All"
msgstr "取消全部"

# ── 二进制预处理: 双模式 + 失败原因透出 (2026-11 新增) ──
# 注: 本批次由人工补录 (本机无 msgmerge), 后续重新生成 .pot 时请逐条核对

#: src/services/binary_converter.vala
msgid "Failed to create tmp dir for PDF conversion"
msgstr "创建 PDF 转换临时目录失败"

#: src/services/binary_converter.vala
#, c-format
msgid "LibreOffice conversion failed (exit %d): %s"
msgstr "LibreOffice 转换失败 (退出码 %d): %s"

#: src/services/binary_converter.vala
msgid "LibreOffice is required to convert Office documents. Download the converter package (see README) or install LibreOffice, then retry."
msgstr "转换 Office 文档需要 LibreOffice:请下载转换器包(见 README)或安装 LibreOffice 后重试。"

#: src/services/binary_converter.vala
#, c-format
msgid "pdftoppm failed: %s"
msgstr "pdftoppm 失败: %s"

#: src/services/binary_preprocessor.vala
msgid "Multimodal AI is disabled or credential is empty, cannot convert binary files"
msgstr "多模态 AI 未启用或凭据为空, 无法转换二进制文件"

#: src/services/vlm_task_runner.vala
msgid "Multimodal AI is disabled in settings"
msgstr "多模态 AI 在设置中未启用"

#: src/services/vlm_task_runner.vala
msgid "Multimodal AI credential is empty, cannot convert binary files"
msgstr "多模态 AI 凭据为空, 无法转换二进制文件"

#: src/services/vlm_task_runner.vala
#, c-format
msgid "Image load failed: %s"
msgstr "图片加载失败: %s"

#: src/services/vlm_task_runner.vala
#, c-format
msgid "Document render failed: %s"
msgstr "文档渲染失败: %s"

#: src/services/office_text_extractor.vala
#, c-format
msgid "File too large for text extraction: %s"
msgstr "文件过大, 无法进行文本抽取: %s"

#: src/services/office_text_extractor.vala
msgid "Unsupported format for text extraction"
msgstr "该格式不支持文本抽取"

#: src/services/office_text_extractor.vala
msgid "Not a valid DOCX (missing word/document.xml)"
msgstr "不是有效的 DOCX (缺少 word/document.xml)"

#: src/services/office_text_extractor.vala
msgid "Not a valid XLSX (no worksheet content)"
msgstr "不是有效的 XLSX (无工作表内容)"

#: src/services/office_text_extractor.vala
msgid "Not a valid PPTX (no slides)"
msgstr "不是有效的 PPTX (无幻灯片)"

#: src/services/office_text_extractor.vala
msgid "Not a valid ODF document (missing content.xml)"
msgstr "不是有效的 ODF 文档 (缺少 content.xml)"

#: src/services/office_text_extractor.vala
#, c-format
msgid "pdftotext failed: %s"
msgstr "pdftotext 失败: %s"

#: src/services/office_text_extractor.vala
msgid "PDF appears to be scanned (no extractable text)"
msgstr "PDF 疑似扫描件 (无可抽取文本)"

#: src/services/office_text_extractor.vala
msgid "Not a ZIP container (EOCD not found)"
msgstr "不是 ZIP 容器 (未找到 EOCD)"

#: src/services/office_text_extractor.vala
msgid "ZIP64 archives are not supported"
msgstr "不支持 ZIP64 归档"

#: src/services/office_text_extractor.vala
msgid "Encrypted ZIP entries are not supported"
msgstr "不支持加密的 ZIP 条目"

#: src/services/office_text_extractor.vala
#, c-format
msgid "Unsupported ZIP compression method %u"
msgstr "不支持的 ZIP 压缩方法 %u"

#: src/services/office_text_extractor.vala
msgid "ZIP deflate decompression failed"
msgstr "ZIP deflate 解压失败"

#: src/services/office_text_extractor.vala
msgid "Corrupted ZIP container"
msgstr "ZIP 容器已损坏"

#: src/services/office_text_extractor.vala
#, c-format
msgid "Text truncated (over %s)"
msgstr "文本已截断 (超过 %s)"

#: src/cli.vala
msgid "Text extracted locally"
msgstr "本地文本抽取完成"

#: src/widgets/preferences_dialog.vala
msgid "Processing Mode"
msgstr "处理方式"

#: src/widgets/preferences_dialog.vala
msgid "Visual conversion (keeps charts, formulas, tables)"
msgstr "视觉转换 (保留图表/公式/表格)"

#: src/widgets/preferences_dialog.vala
msgid "Text extraction (fast, offline, text only)"
msgstr "文本抽取 (快速离线, 仅文本)"

#: src/widgets/preferences_dialog.vala
msgid "Visual conversion renders documents to images and needs LibreOffice for Office files; text extraction parses Office/PDF locally without AI credentials."
msgstr "视觉转换把文档渲染为图片, Office 文件需要 LibreOffice; 文本抽取在本地直接解析 Office/PDF, 无需 AI 凭据。"

#: src/window.vala
#, c-format
msgid "Reason: %s"
msgstr "原因: %s"
Loading
Loading