14 Commits

Author SHA1 Message Date
mamamiyear baec01610c feat: add screenshot OCR extraction 2026-07-08 00:42:49 +08:00
mamamiyear f90612976f feat: add text annotation tool 2026-07-07 23:49:24 +08:00
mamamiyear a15f5b2b78 refactor: localize tray menu items to Chinese
Rename the menu bar items to 截图 / 设置 / 退出 for the primary Chinese
audience.
2026-07-07 16:56:48 +08:00
mamamiyear cd3d045ef5 refactor: localize settings sidebar tab names to Chinese
Rename the settings tabs to 通用设置 / 对象存储 / 远程主机 / 智能识图 so
the sidebar reads naturally for the primary Chinese audience.
2026-07-07 16:51:39 +08:00
mamamiyear 3aad93e7e8 feat: copy saved file path to clipboard after local save
After writing the screenshot to disk, copy its path to the clipboard so
it can be pasted immediately, matching the upload and summary flows. The
save status now shows the copied path. A clipboard error does not fail
the save itself.
2026-07-07 16:37:47 +08:00
mamamiyear 85d5fcc722 build: silence duplicate -lobjc linker warning on Xcode 15+
Add -Wl,-no_warn_duplicate_libraries to the darwin cgo LDFLAGS and allow
it through the cgo flag allowlist via CGO_LDFLAGS_ALLOW in the build
scripts, so the ld-prime duplicate -lobjc note no longer clutters build
output.
2026-07-07 14:16:24 +08:00
mamamiyear 778055cd1d feat: unify status feedback for copy, save and cancel actions
Route the copy-to-clipboard and save-to-disk pipelines through
emitOperationStatus so they show the same running/success/error stages
as the upload, save-remote and summary flows, and surface a short
"已取消" status when a capture is cancelled. This gives every capture
action a consistent HUD experience.
2026-07-07 14:03:02 +08:00
mamamiyear 87874e64cf feat: prefer base64 inline image for summary, fall back to S3
Send the screenshot to the multimodal model inline as a base64 data URL
so summarisation no longer requires S3 to be configured or publicly
reachable. Only when the PNG exceeds the provider MaxInlineBytes limit
do we upload to S3, verify the public URL is reachable, and delete the
temporary object once done. Oversized images without S3 now surface a
clear hint instead of a hard S3 requirement.
2026-07-07 13:39:23 +08:00
mamamiyear 7c53c0f184 fix(hotkey): split platform files by platform+cgo build matrix
The `!darwin` guard on the modifier helpers was too broad: golang.design/x/
hotkey only exposes its Mod*/Key* constants from the darwin&&cgo, linux&&cgo
and windows backends, while the `!windows && !cgo` complement (e.g. Linux
cross-builds with CGO_ENABLED=0) falls back to a stub with no constants. That
made hk.ModCtrl/ModShift/KeyA... undefined and produced editor/vet warnings.

Realign the build tags with the upstream matrix and confine every hk.* constant
to a tagged file so the shared hotkey.go compiles everywhere. Also fix the
Linux mapping (option -> hk.Mod1; hk.ModAlt does not exist on X11).
2026-07-07 10:29:59 +08:00
mamamiyear 5ce3eba455 docs: add AGENTS.md architecture overview 2026-07-07 10:29:59 +08:00
mamamiyear 71ca67c0ee refactor: change default size of settings panel 2026-07-07 01:12:18 +08:00
mamamiyear 777d5cb6c3 refactor: change dark theme color 2026-07-07 01:09:22 +08:00
mamamiyear 712a2cbb6a feat: you can summary screen shot directly 2026-07-07 00:59:26 +08:00
mamamiyear 39f0aaae02 refactor: merge mark and action toolbars to the right side
Place the mark toolbar to the left of the action toolbar on the same row
with an 8px gap, so the two button groups never overlap on tiny selections.
2026-06-14 14:30:43 +08:00
39 changed files with 3854 additions and 156 deletions
+67
View File
@@ -0,0 +1,67 @@
# AGENTS.md
## 项目概述
SnapGo 是一个常驻菜单栏的轻量截图工具:全局快捷键唤起选区截图,确认后自动上传到 S3 兼容对象存储(或通过 SSH/SCP 保存到远端),并把可分享链接复制到剪贴板。目前主要打磨 macOS 体验。
技术栈:**Wails v2**Go 后端 + WebView 前端)+ **Vue 3 / TypeScript / Vite** 前端。Go 后端遵循分层架构(domain → application → infrastructure),macOS 上截图选区使用原生 AppKit overlay,其它平台回退到 Wails 透明窗口 overlay。
### 分层架构(`internal/`
- `domain/`:核心业务类型与接口,**不依赖任何第三方 SDK 或 OS API**(仅标准库)。如 `OSSProvider` 接口、`AppConfig`/`S3Config`/`SSHConfig``Screenshot`/`UploadResult`
- `application/`:用例编排(capture → upload/save → clipboard → notify)。**只依赖 domain 接口**,具体适配器从 `main.go`/`app.go` 注入,便于单测。如 `CaptureAndUploadService``CaptureActionsService``CaptureAndSSHService``ApplyAnnotations`
- `infrastructure/`:对接外部世界的适配器实现 —— `oss/`(S3)、`ssh/`(内置 Go client + Kerberos 委托系统 ssh)、`clipboard/``config/`(JSON 文件存储)、`screencapture/``display/``cursor/``hotkey/``tray/`(菜单栏)、`logging/`
### 顶层文件(`main` 包,根目录)
- `main.go`Wails 启动入口,embed `frontend/dist`,装配 tray 与 `App`
- `app.go``App` 结构体,通过 Wails `Bind` 暴露给前端的方法(RPC),编排截图/上传/保存流程与 overlay 窗口生命周期。
- `*_darwin.go` / `*_other.go`:平台分离的 overlay、dock 图标、激活策略、帧刷新等实现。`native_overlay_darwin.go` 使用 cgo + AppKit。
### 关键约定
- 前端通过 Wails 自动生成的 `frontend/wailsjs/go/main/App.js` 调用后端 `App` 的导出方法;后端通过 `wruntime.EventsEmit` 向前端推事件(如 `upload:success``upload:failure``hotkey:ready``capture:overlay`)。
- 新增 OSS provider 只需在 `infrastructure/oss/` 加适配器并实现 `domain.OSSProvider`,无需改 application 层。
- S3 与 SSH 走独立 pipeline`runUploadPipeline` vs `runSaveRemotePipeline`),避免 provider 分支泄漏进彼此。
## 构建与命令
依赖前置:Go 1.23+、Node/npm、`wails` CLI`go install github.com/wailsapp/wails/v2/cmd/wails@latest`)。
- `go mod tidy`:同步 Go 依赖。
- `wails dev`:开发模式(热重载前端 + 后端),日志可直接在终端看到。
- `wails build -platform darwin/arm64 -clean`:构建 `.app`(产物在 `build/bin/`)。
- `./scripts/create-dev-cert.sh`:首次创建本地自签开发证书(保证 macOS TCC 权限不被反复重置)。
- `./scripts/dev-build.sh`:本地迭代用 —— 构建 + 稳定证书重签名 + 同步到 `/Applications` + 打开。优先用它而非裸 `wails build`
- `./scripts/release.sh`:一键 build → sign → DMG → notarizenotarize 需 `NOTARIZE=1` 与凭证,默认跳过)。`ARCH=universal` 可出 fat binary。
- 前端单独命令(`frontend/`):`npm run dev` / `npm run build``vue-tsc --noEmit && vite build`/ `npm run preview`
## 代码风格
- **Go**:标准 `gofmt`(tab 缩进)。包级 doc 注释说明“设计动机/rationale”是本仓库的强约定,注释解释“为什么”而非“是什么”。导出标识符必须有注释。错误用 `fmt.Errorf("...: %w", err)` 包装。日志统一用 `log/slog`(结构化 kv,如 `slog.Info("...", "host", h)`)。
- **平台分离**:用 `_darwin.go` / `_other.go` 文件名后缀或 `//go:build` 标签隔离平台代码,保持跨平台编译通过。
- **TypeScript/Vue**Vue 3 `<script setup>` + Composition API2 空格缩进。视图在 `frontend/src/views/`,组件在 `components/`
- 注释中可使用中文(部分 infra 包如 logging 已用中文 rationale),与现有文件保持一致即可。
- 不要手改 `frontend/wailsjs/` 下文件,它们由 Wails 生成。
## 测试
- 框架:Go 标准 `testing` 包(无第三方断言库)。运行:`go test ./...`
- 现有测试集中在 `internal/application/`(如 `capture_actions_test.go``annotation_test.go`)。
- 约定:application 层依赖 domain 接口,测试用手写 fake 实现(如 `fakeClipboard``fakeNotifier`)注入,不触碰真实 OS/网络。文件 IO 测试使用 `t.TempDir()`
- infrastructure 适配器(S3/SSH/screencapture 等)依赖真实环境,通常不写单测;SSH 连通性可借助 `cmd/sshdiag/` 诊断工具与设置页的 “Test connection” 按钮验证。
## 安全
- **凭证落盘**:配置(含 S3 access key、SSH 密码)以 JSON 存于 `os.UserConfigDir()/SnapGo/config.json`,文件权限 `0600`、目录 `0700`。尚未接入 OS keychain(后续计划),改动配置存储时务必保持严格权限并避免把密钥写入日志。
- **日志**:写入 `~/Library/Logs/SnapGo/snapgo.log`(大小滚动,5MiB×2 备份)。记录连接信息时只记 host/user/port 等非敏感字段,**切勿记录密码/密钥**(参考 `app.go` 中的 `slog` 用法)。
- **SSH**`StrictHostKey=false` 时使用 `InsecureIgnoreHostKey()`(牺牲安全换首启可用性,仅限个人 LAN);Kerberos 模式委托系统 `/usr/bin/ssh` 复用 `kinit` 凭证缓存。`PathPrefix` 相对远端 `$HOME`,会剥离前导 `/``~` 防止逃逸。
- **上传兜底**:上传失败时把 PNG 落地到 `~/Pictures/SnapGo` 并复制本地路径,保证不丢截图。
- macOS 需 Screen Recording / AccessibilityTCC)权限;稳定的代码签名身份与安装路径是权限保持的关键(见 `dev-build.sh` 注释)。
## 配置
- Wails 项目配置:`wails.json`(产物名、frontend install/build/dev 命令、应用元信息)。
- 运行时配置:首启写入默认值(`domain.DefaultAppConfig()`:快捷键 `cmd+shift+a`、S3 `pathPrefix=snapgo/`、SSH port 22 / timeout 10s)。原子写入(temp + rename)。
- 环境变量:`SNAPGO_LOG_LEVEL=debug|info|warn|error`(默认 debug)。Release 相关:`ARCH``NOTARIZE``SIGN_MODE``DEVELOPER_ID_APPLICATION``KEYCHAIN_PROFILE``APPLE_ID`/`APPLE_TEAM_ID`/`APPLE_APP_SPECIFIC_PASSWORD`
- S3 兼容性:`UsePathStyle` 默认 true(适配 MinIO/R2);`PublicURLBase` 可选用于 CDN/自定义域名,否则回退 `{Endpoint}/{Bucket}/{Key}`
+6
View File
@@ -5,6 +5,12 @@ package main
/* /*
#cgo CFLAGS: -x objective-c -fobjc-arc #cgo CFLAGS: -x objective-c -fobjc-arc
#cgo LDFLAGS: -framework AppKit #cgo LDFLAGS: -framework AppKit
// Silence the "ignoring duplicate libraries: '-lobjc'" note emitted by the
// Xcode 15+ linker (ld-prime): cgo links -lobjc for our Objective-C code and
// the Go darwin runtime pulls it in as well, which the new linker flags as a
// duplicate. Suppressing it here keeps the build output clean; the flag is a
// no-op on the legacy ld64 path.
#cgo LDFLAGS: -Wl,-no_warn_duplicate_libraries
#include <dispatch/dispatch.h> #include <dispatch/dispatch.h>
#import <AppKit/AppKit.h> #import <AppKit/AppKit.h>
+328 -8
View File
@@ -22,6 +22,8 @@ import (
"github.com/mmmy/snapgo/internal/infrastructure/config" "github.com/mmmy/snapgo/internal/infrastructure/config"
"github.com/mmmy/snapgo/internal/infrastructure/display" "github.com/mmmy/snapgo/internal/infrastructure/display"
"github.com/mmmy/snapgo/internal/infrastructure/hotkey" "github.com/mmmy/snapgo/internal/infrastructure/hotkey"
llmpkg "github.com/mmmy/snapgo/internal/infrastructure/llm"
ocrpkg "github.com/mmmy/snapgo/internal/infrastructure/ocr"
"github.com/mmmy/snapgo/internal/infrastructure/oss" "github.com/mmmy/snapgo/internal/infrastructure/oss"
"github.com/mmmy/snapgo/internal/infrastructure/screencapture" "github.com/mmmy/snapgo/internal/infrastructure/screencapture"
sshpkg "github.com/mmmy/snapgo/internal/infrastructure/ssh" sshpkg "github.com/mmmy/snapgo/internal/infrastructure/ssh"
@@ -96,6 +98,15 @@ type CaptureActionResult struct {
Path string `json:"path,omitempty"` Path string `json:"path,omitempty"`
} }
// OperationStatusPayload drives the frontend status HUD and mirrors the
// native macOS HUD state.
type OperationStatusPayload struct {
Operation string `json:"operation"`
Phase string `json:"phase"`
Message string `json:"message"`
State string `json:"state"`
}
// NewApp creates a new App with collaborators already initialised. // NewApp creates a new App with collaborators already initialised.
func NewApp() *App { func NewApp() *App {
store, err := config.NewFileStore() store, err := config.NewFileStore()
@@ -110,6 +121,7 @@ func NewApp() *App {
slog.Warn("config load failed, using defaults", "err", lerr) slog.Warn("config load failed, using defaults", "err", lerr)
} }
} }
cfg.Normalize()
return &App{ return &App{
cfg: cfg, cfg: cfg,
configFile: store, configFile: store,
@@ -227,8 +239,8 @@ func (a *App) runInteractiveCapture() {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
const ( const (
settingsWidth = 1080 settingsWidth = 1000
settingsHeight = 720 settingsHeight = 820
) )
// showOverlayWindow prepares the hidden main window as a borderless, // showOverlayWindow prepares the hidden main window as a borderless,
@@ -297,23 +309,44 @@ func (a *App) runUploadPipeline(provider domain.OSSProvider, pngBytes []byte) er
FallbackDir: filepath.Join(userPicturesDir(), "SnapGo"), FallbackDir: filepath.Join(userPicturesDir(), "SnapGo"),
PathPrefix: cfg.S3.PathPrefix, PathPrefix: cfg.S3.PathPrefix,
} }
return svc.ExecuteWithBytes(a.ctx, pngBytes) a.emitOperationStatus("upload", "上传中", "正在上传截图到 S3", "running")
if err := svc.ExecuteWithBytes(a.ctx, pngBytes); err != nil {
a.emitOperationStatus("upload", "上传失败", err.Error(), "error")
return err
}
a.emitOperationStatus("upload", "上传完成", "链接已复制到剪贴板", "success")
return nil
} }
func (a *App) runCopyImagePipeline(pngBytes []byte) error { func (a *App) runCopyImagePipeline(pngBytes []byte) error {
svc := &application.CaptureActionsService{ svc := &application.CaptureActionsService{
Clipboard: a.clip, Clipboard: a.clip,
Notifier: &runtimeNotifier{ctx: a.ctx},
} }
return svc.CopyImage(a.ctx, pngBytes) a.emitOperationStatus("copy", "复制中", "正在把截图复制到剪贴板", "running")
if err := svc.CopyImage(a.ctx, pngBytes); err != nil {
a.emitOperationStatus("copy", "复制失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
a.emitOperationStatus("copy", "复制完成", "截图已复制到剪贴板", "success")
wruntime.EventsEmit(a.ctx, "upload:success", "image copied to clipboard")
return nil
} }
func (a *App) runSaveImagePipeline(pngBytes []byte, dir string) (string, error) { func (a *App) runSaveImagePipeline(pngBytes []byte, dir string) (string, error) {
svc := &application.CaptureActionsService{ svc := &application.CaptureActionsService{
Clipboard: a.clip, Clipboard: a.clip,
Notifier: &runtimeNotifier{ctx: a.ctx},
} }
return svc.SaveImage(a.ctx, pngBytes, dir) a.emitOperationStatus("save", "保存中", "正在保存截图到本地", "running")
path, err := svc.SaveImage(a.ctx, pngBytes, dir)
if err != nil {
a.emitOperationStatus("save", "保存失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return "", err
}
a.emitOperationStatus("save", "保存完成", "文件路径已复制到剪贴板:"+path, "success")
wruntime.EventsEmit(a.ctx, "upload:success", path)
return path, nil
} }
// runSaveRemotePipeline uploads the captured PNG to the configured SSH host // runSaveRemotePipeline uploads the captured PNG to the configured SSH host
@@ -332,6 +365,7 @@ func (a *App) runSaveRemotePipeline(pngBytes []byte) error {
err := fmt.Errorf("SSH host/user is not configured") err := fmt.Errorf("SSH host/user is not configured")
slog.Warn("save-remote rejected: ssh not configured", slog.Warn("save-remote rejected: ssh not configured",
"host", cfg.SSH.Host, "user", cfg.SSH.User) "host", cfg.SSH.Host, "user", cfg.SSH.User)
a.emitOperationStatus("save-remote", "需要配置 SSH", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error()) wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err return err
} }
@@ -355,7 +389,157 @@ func (a *App) runSaveRemotePipeline(pngBytes []byte) error {
Notifier: &runtimeNotifier{ctx: a.ctx}, Notifier: &runtimeNotifier{ctx: a.ctx},
Cfg: cfg.SSH, Cfg: cfg.SSH,
} }
return svc.ExecuteWithBytes(a.ctx, pngBytes) a.emitOperationStatus("save-remote", "保存中", "正在保存截图到远端 SSH", "running")
if err := svc.ExecuteWithBytes(a.ctx, pngBytes); err != nil {
a.emitOperationStatus("save-remote", "保存失败", err.Error(), "error")
return err
}
a.emitOperationStatus("save-remote", "保存完成", "远端路径已复制到剪贴板", "success")
return nil
}
func (a *App) runSummaryPipeline(pngBytes []byte) error {
a.mu.RLock()
cfg := a.cfg
a.mu.RUnlock()
if !cfg.IsLLMConfigured() {
err := fmt.Errorf("请先在 LLM 配置页填写 API Key 和模型")
a.emitOperationStatus("summary", "需要配置 LLM", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
providerID, llmCfg, _ := cfg.ActiveLLMProvider()
visionClient, err := llmpkg.NewVisionClient(providerID, llmCfg)
if err != nil {
a.emitOperationStatus("summary", "LLM 配置错误", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
svc := &application.CaptureSummaryService{
Summarizer: visionClient,
Clipboard: a.clip,
Prompt: cfg.LLM.Prompt,
PathPrefix: cfg.S3.PathPrefix,
MaxInlineBytes: llmCfg.MaxInlineBytes,
}
// Preferred path: send the screenshot inline as a base64 data URL so the
// summary never depends on S3 being configured or publicly reachable.
if svc.CanInline(pngBytes) {
a.emitOperationStatus("summary", "识别中", "正在请求多模态模型(本地内联图片)", "running")
summary, err := svc.Summarize(a.ctx, svc.InlineDataURL(pngBytes))
if err != nil {
a.emitOperationStatus("summary", "识别失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
return a.finishSummary(svc, summary)
}
// Oversized image: fall back to uploading to S3 and passing a public URL.
if !cfg.IsS3Configured() {
err := fmt.Errorf("截图尺寸超过内联上限(%d 字节),请缩小截图区域,或在 S3 配置页填写公网可达的对象存储后重试", llmCfg.MaxInlineBytes)
a.emitOperationStatus("summary", "尺寸超限", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
s3Provider, err := oss.NewS3Provider(cfg.S3)
if err != nil {
a.emitOperationStatus("summary", "S3 配置错误", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
svc.Provider = s3Provider
a.emitOperationStatus("summary", "上传中", "截图较大,正在上传供模型读取", "running")
uploaded, err := svc.UploadImage(a.ctx, pngBytes)
if err != nil {
a.emitOperationStatus("summary", "上传失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
// The remote model fetches the URL itself, so verify it is reachable and
// clean up the temporary object once we are done with it either way.
defer func() {
if derr := svc.DeleteUploaded(a.ctx, uploaded.Key); derr != nil {
slog.Warn("summary: failed to delete temporary object", "key", uploaded.Key, "err", derr)
}
}()
if err := svc.EnsureReachable(a.ctx, uploaded.URL); err != nil {
a.emitOperationStatus("summary", "链接不可达", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
a.emitOperationStatus("summary", "识别中", "图片已上传,正在请求多模态模型", "running")
summary, err := svc.Summarize(a.ctx, uploaded.URL)
if err != nil {
a.emitOperationStatus("summary", "识别失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
return a.finishSummary(svc, summary)
}
// finishSummary copies the summary to the clipboard and emits the terminal
// success/failure UI state shared by the inline and S3 paths.
func (a *App) finishSummary(svc *application.CaptureSummaryService, summary string) error {
if err := svc.CopySummary(a.ctx, summary); err != nil {
a.emitOperationStatus("summary", "复制失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
a.emitOperationStatus("summary", "识别完成", "总结已复制到剪贴板", "success")
wruntime.EventsEmit(a.ctx, "upload:success", "summary copied to clipboard")
return nil
}
func (a *App) runOCRPipeline(pngBytes []byte) error {
a.mu.RLock()
cfg := a.cfg
a.mu.RUnlock()
if !cfg.IsOCRConfigured() {
err := fmt.Errorf("请先在文字提取配置页填写 OCR Provider 的 AccessKey ID 和 AccessKey Secret")
a.emitOperationStatus("ocr", "需要配置 OCR", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
providerID, ocrCfg, _ := cfg.ActiveOCRProvider()
recognizer, err := ocrpkg.NewClient(providerID, ocrCfg)
if err != nil {
a.emitOperationStatus("ocr", "OCR 配置错误", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
svc := &application.CaptureOCRService{
Recognizer: recognizer,
Clipboard: a.clip,
}
a.emitOperationStatus("ocr", "识别中", "正在提取截图文字", "running")
text, err := svc.Recognize(a.ctx, pngBytes)
if err != nil {
a.emitOperationStatus("ocr", "识别失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
if err := svc.CopyText(a.ctx, text); err != nil {
a.emitOperationStatus("ocr", "复制失败", err.Error(), "error")
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
a.emitOperationStatus("ocr", "提取完成", "文字已复制到剪贴板", "success")
wruntime.EventsEmit(a.ctx, "upload:success", "ocr text copied to clipboard")
return nil
} }
func (a *App) consumePendingCapture() (*pendingCapture, error) { func (a *App) consumePendingCapture() (*pendingCapture, error) {
@@ -408,6 +592,7 @@ func (a *App) GetConfig() domain.AppConfig {
// SaveConfig persists the supplied configuration and re-registers the hotkey // SaveConfig persists the supplied configuration and re-registers the hotkey
// if it changed. // if it changed.
func (a *App) SaveConfig(cfg domain.AppConfig) error { func (a *App) SaveConfig(cfg domain.AppConfig) error {
cfg.Normalize()
a.mu.Lock() a.mu.Lock()
prev := a.cfg prev := a.cfg
a.cfg = cfg a.cfg = cfg
@@ -705,6 +890,64 @@ func (a *App) SaveRegionToRemote(result CaptureResult) error {
return a.runSaveRemotePipeline(cropped) return a.runSaveRemotePipeline(cropped)
} }
// SummarizeRegion uploads the selected screenshot to S3, sends the public URL
// to the configured multimodal LLM, and copies the resulting summary.
func (a *App) SummarizeRegion(result CaptureResult) error {
pc, err := a.consumePendingCapture()
if err != nil {
slog.Warn("SummarizeRegion: no pending capture", "err", err)
return err
}
defer func() {
a.capturing.Store(false)
a.dismissOverlay()
}()
a.dismissOverlay()
flushFrame()
slog.Info("SummarizeRegion: capturing region",
"x", result.Rect.X, "y", result.Rect.Y,
"w", result.Rect.W, "h", result.Rect.H,
"annotations", len(result.Annotations))
cropped, err := a.captureSelectedPNG(result, pc)
if err != nil {
slog.Error("SummarizeRegion: capture failed", "err", err)
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
return a.runSummaryPipeline(cropped)
}
// ExtractTextRegion sends the selected screenshot to the configured OCR
// provider and copies the extracted text to the clipboard.
func (a *App) ExtractTextRegion(result CaptureResult) error {
pc, err := a.consumePendingCapture()
if err != nil {
slog.Warn("ExtractTextRegion: no pending capture", "err", err)
return err
}
defer func() {
a.capturing.Store(false)
a.dismissOverlay()
}()
a.dismissOverlay()
flushFrame()
slog.Info("ExtractTextRegion: capturing region",
"x", result.Rect.X, "y", result.Rect.Y,
"w", result.Rect.W, "h", result.Rect.H,
"annotations", len(result.Annotations))
cropped, err := a.captureSelectedPNG(result, pc)
if err != nil {
slog.Error("ExtractTextRegion: capture failed", "err", err)
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
return a.runOCRPipeline(cropped)
}
// SaveNativeRegionToRemote is the macOS-native overlay equivalent of // SaveNativeRegionToRemote is the macOS-native overlay equivalent of
// SaveRegionToRemote. The AppKit panel is already closed by the time this // SaveRegionToRemote. The AppKit panel is already closed by the time this
// runs (see saveRemoteSelection in native_overlay_darwin.go), so we only // runs (see saveRemoteSelection in native_overlay_darwin.go), so we only
@@ -735,6 +978,60 @@ func (a *App) SaveNativeRegionToRemote(result CaptureResult) error {
return a.runSaveRemotePipeline(cropped) return a.runSaveRemotePipeline(cropped)
} }
// SummarizeNativeRegion is the macOS-native overlay equivalent of
// SummarizeRegion. The AppKit panel is already closed before this runs.
func (a *App) SummarizeNativeRegion(result CaptureResult) error {
pc, err := a.consumePendingCapture()
if err != nil {
slog.Warn("SummarizeNativeRegion: no pending capture", "err", err)
return err
}
defer func() {
a.capturing.Store(false)
hideDockIcon()
}()
flushFrame()
slog.Info("SummarizeNativeRegion: capturing region",
"x", result.Rect.X, "y", result.Rect.Y,
"w", result.Rect.W, "h", result.Rect.H,
"annotations", len(result.Annotations))
cropped, err := a.captureSelectedPNG(result, pc)
if err != nil {
slog.Error("SummarizeNativeRegion: capture failed", "err", err)
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
return a.runSummaryPipeline(cropped)
}
// ExtractTextNativeRegion is the macOS-native overlay equivalent of
// ExtractTextRegion. The AppKit panel is already closed before this runs.
func (a *App) ExtractTextNativeRegion(result CaptureResult) error {
pc, err := a.consumePendingCapture()
if err != nil {
slog.Warn("ExtractTextNativeRegion: no pending capture", "err", err)
return err
}
defer func() {
a.capturing.Store(false)
hideDockIcon()
}()
flushFrame()
slog.Info("ExtractTextNativeRegion: capturing region",
"x", result.Rect.X, "y", result.Rect.Y,
"w", result.Rect.W, "h", result.Rect.H,
"annotations", len(result.Annotations))
cropped, err := a.captureSelectedPNG(result, pc)
if err != nil {
slog.Error("ExtractTextNativeRegion: capture failed", "err", err)
wruntime.EventsEmit(a.ctx, "upload:failure", err.Error())
return err
}
return a.runOCRPipeline(cropped)
}
func parseNativeAnnotations(raw string) []application.Annotation { func parseNativeAnnotations(raw string) []application.Annotation {
if raw == "" { if raw == "" {
return nil return nil
@@ -756,6 +1053,7 @@ func (a *App) CancelRegion() {
a.pendingMu.Unlock() a.pendingMu.Unlock()
a.capturing.Store(false) a.capturing.Store(false)
a.dismissOverlay() a.dismissOverlay()
a.emitOperationStatus("capture", "已取消", "本次截图已取消", "success")
} }
// CancelNativeRegion releases native-overlay state without touching the // CancelNativeRegion releases native-overlay state without touching the
@@ -766,6 +1064,7 @@ func (a *App) CancelNativeRegion() {
a.pendingMu.Unlock() a.pendingMu.Unlock()
a.capturing.Store(false) a.capturing.Store(false)
hideDockIcon() hideDockIcon()
a.emitOperationStatus("capture", "已取消", "本次截图已取消", "success")
} }
// ShowWindow brings the main window back to the foreground in its normal // ShowWindow brings the main window back to the foreground in its normal
@@ -784,6 +1083,27 @@ func (a *App) QuitApp() {
os.Exit(0) os.Exit(0)
} }
func (a *App) emitOperationStatus(operation, phase, message, state string) {
if a.ctx != nil {
wruntime.EventsEmit(a.ctx, "operation:status", OperationStatusPayload{
Operation: operation,
Phase: phase,
Message: message,
State: state,
})
}
switch state {
case "success":
showOperationStatus(phase, message, operationStatusSuccess)
hideOperationStatusAfter(1400 * time.Millisecond)
case "error":
showOperationStatus(phase, message, operationStatusError)
hideOperationStatusAfter(2600 * time.Millisecond)
default:
showOperationStatus(phase, message, operationStatusRunning)
}
}
// runtimeNotifier emits success / failure events via the Wails runtime. // runtimeNotifier emits success / failure events via the Wails runtime.
type runtimeNotifier struct{ ctx context.Context } type runtimeNotifier struct{ ctx context.Context }
+238 -19
View File
@@ -3,9 +3,7 @@
* App shell — switches between two distinct UI modes that share the same * App shell — switches between two distinct UI modes that share the same
* Wails window: * Wails window:
* *
* • "settings" : full settings UI (self-drawn title bar + sidebar + form). * • "settings" : full settings UI (native title bar + sidebar + form).
* Self-drawn because the window is now Frameless to make
* the overlay paint edge-to-edge.
* • "overlay" : Snipaste-style region picker that fills the whole * • "overlay" : Snipaste-style region picker that fills the whole
* primary display. * primary display.
* *
@@ -13,7 +11,7 @@
* emits a `capture:overlay` event with the screenshot payload so the * emits a `capture:overlay` event with the screenshot payload so the
* frontend knows when (and what) to render. * frontend knows when (and what) to render.
*/ */
import { onMounted, onUnmounted, ref } from 'vue' import { computed, onMounted, onUnmounted, ref } from 'vue'
import SettingsView from './views/SettingsView.vue' import SettingsView from './views/SettingsView.vue'
import Toast from './components/Toast.vue' import Toast from './components/Toast.vue'
import CaptureOverlay from './views/CaptureOverlay.vue' import CaptureOverlay from './views/CaptureOverlay.vue'
@@ -24,7 +22,10 @@ import {
CopyRegionImage, CopyRegionImage,
SaveRegionImage, SaveRegionImage,
SaveRegionToRemote, SaveRegionToRemote,
SummarizeRegion,
ExtractTextRegion,
CancelRegion, CancelRegion,
GetConfig,
} from '../wailsjs/go/main/App' } from '../wailsjs/go/main/App'
type HotkeyStatus = type HotkeyStatus =
@@ -36,18 +37,39 @@ const hotkeyStatus = ref<HotkeyStatus>({ state: 'unknown' })
type Mode = 'settings' | 'overlay' type Mode = 'settings' | 'overlay'
const mode = ref<Mode>('settings') const mode = ref<Mode>('settings')
type ThemeMode = 'auto' | 'light' | 'dark'
const themeMode = ref<ThemeMode>('auto')
const appThemeClass = computed(() => `theme-${themeMode.value}`)
function normalizeTheme(theme: unknown): ThemeMode {
return theme === 'light' || theme === 'dark' || theme === 'auto'
? theme
: 'auto'
}
async function loadThemePreference() {
try {
const cfg = await GetConfig()
themeMode.value = normalizeTheme((cfg as any)?.theme)
} catch {
themeMode.value = 'auto'
}
}
// `SettingsTab` is hoisted to the App shell so the sidebar (which lives // `SettingsTab` is hoisted to the App shell so the sidebar (which lives
// here) and the inner SettingsView (which renders the matching card) can // here) and the inner SettingsView (which renders the matching card) can
// share a single source of truth without an event bus. // share a single source of truth without an event bus.
type SettingsTab = 'general' | 's3' | 'ssh' type SettingsTab = 'general' | 's3' | 'ssh' | 'llm' | 'ocr'
const activeTab = ref<SettingsTab>('general') const activeTab = ref<SettingsTab>('general')
// Sidebar entries are declarative so adding a destination type later is // Sidebar entries are declarative so adding a destination type later is
// a one-line change. The label values match the user-facing tab names. // a one-line change. The label values match the user-facing tab names.
const sidebarItems: Array<{ id: SettingsTab; label: string }> = [ const sidebarItems: Array<{ id: SettingsTab; label: string }> = [
{ id: 'general', label: 'General' }, { id: 'general', label: '通用设置' },
{ id: 's3', label: 'S3' }, { id: 's3', label: '对象存储' },
{ id: 'ssh', label: 'SSH' }, { id: 'ssh', label: '远程主机' },
{ id: 'llm', label: '智能识图' },
{ id: 'ocr', label: '文字提取' },
] ]
interface OverlayPayload { interface OverlayPayload {
@@ -62,6 +84,14 @@ const capturing = ref(false)
const toast = ref<{ kind: 'success' | 'error'; text: string } | null>(null) const toast = ref<{ kind: 'success' | 'error'; text: string } | null>(null)
let toastTimer: number | undefined let toastTimer: number | undefined
const operationStatus = ref<{
operation: string
phase: string
message: string
state: 'running' | 'success' | 'error'
} | null>(null)
let operationTimer: number | undefined
function showToast(kind: 'success' | 'error', text: string) { function showToast(kind: 'success' | 'error', text: string) {
toast.value = { kind, text } toast.value = { kind, text }
window.clearTimeout(toastTimer) window.clearTimeout(toastTimer)
@@ -70,6 +100,21 @@ function showToast(kind: 'success' | 'error', text: string) {
}, 3000) }, 3000)
} }
function updateOperationStatus(payload: {
operation: string
phase: string
message: string
state: 'running' | 'success' | 'error'
}) {
operationStatus.value = payload
window.clearTimeout(operationTimer)
if (payload.state !== 'running') {
operationTimer = window.setTimeout(() => {
operationStatus.value = null
}, payload.state === 'success' ? 1600 : 3200)
}
}
async function retryHotkey() { async function retryHotkey() {
try { try {
await RetryRegisterHotkey() await RetryRegisterHotkey()
@@ -168,6 +213,50 @@ async function onOverlaySaveRemote(rect: {
} }
} }
async function onOverlaySummarize(rect: {
rect: {
x: number
y: number
w: number
h: number
}
annotations: Array<{
tool: string
color: string
points: Array<{ x: number; y: number }>
}>
}) {
mode.value = 'settings'
overlayPayload.value = null
try {
await SummarizeRegion(rect as any)
} catch {
/* Surfaced via upload:failure */
}
}
async function onOverlayOCR(rect: {
rect: {
x: number
y: number
w: number
h: number
}
annotations: Array<{
tool: string
color: string
points: Array<{ x: number; y: number }>
}>
}) {
mode.value = 'settings'
overlayPayload.value = null
try {
await ExtractTextRegion(rect as any)
} catch {
/* Surfaced via upload:failure */
}
}
async function onOverlayCancel() { async function onOverlayCancel() {
mode.value = 'settings' mode.value = 'settings'
overlayPayload.value = null overlayPayload.value = null
@@ -179,6 +268,7 @@ async function onOverlayCancel() {
} }
onMounted(() => { onMounted(() => {
void loadThemePreference()
EventsOn('capture:start', () => { EventsOn('capture:start', () => {
capturing.value = true capturing.value = true
}) })
@@ -194,7 +284,14 @@ onMounted(() => {
capturing.value = false capturing.value = false
}) })
EventsOn('upload:success', (url: string) => { EventsOn('upload:success', (url: string) => {
showToast('success', `Copied: ${url}`) showToast(
'success',
url === 'summary copied to clipboard'
? 'Summary copied to clipboard'
: url === 'ocr text copied to clipboard'
? 'OCR text copied to clipboard'
: `Copied: ${url}`
)
}) })
EventsOn('upload:failure', (reason: string) => { EventsOn('upload:failure', (reason: string) => {
showToast('error', reason) showToast('error', reason)
@@ -205,6 +302,7 @@ onMounted(() => {
EventsOn('hotkey:error', (reason: string) => { EventsOn('hotkey:error', (reason: string) => {
hotkeyStatus.value = { state: 'error', reason } hotkeyStatus.value = { state: 'error', reason }
}) })
EventsOn('operation:status', updateOperationStatus)
}) })
onUnmounted(() => { onUnmounted(() => {
@@ -216,6 +314,7 @@ onUnmounted(() => {
EventsOff('upload:failure') EventsOff('upload:failure')
EventsOff('hotkey:ready') EventsOff('hotkey:ready')
EventsOff('hotkey:error') EventsOff('hotkey:error')
EventsOff('operation:status')
}) })
</script> </script>
@@ -229,11 +328,13 @@ onUnmounted(() => {
@copy="onOverlayCopy" @copy="onOverlayCopy"
@save="onOverlaySave" @save="onOverlaySave"
@save-remote="onOverlaySaveRemote" @save-remote="onOverlaySaveRemote"
@summarize="onOverlaySummarize"
@ocr="onOverlayOCR"
@cancel="onOverlayCancel" @cancel="onOverlayCancel"
/> />
<!-- Settings mode: standard desktop window; macOS title bar is native. --> <!-- Settings mode: standard desktop window; macOS title bar is native. -->
<div v-else class="app-root"> <div v-else class="app-root" :class="appThemeClass">
<div v-if="hotkeyStatus.state === 'error'" class="permission-banner"> <div v-if="hotkeyStatus.state === 'error'" class="permission-banner">
<div> <div>
<strong>Global hotkey is not active.</strong> <strong>Global hotkey is not active.</strong>
@@ -259,11 +360,29 @@ onUnmounted(() => {
</div> </div>
</aside> </aside>
<section class="content"> <section class="content">
<SettingsView :tab="activeTab" /> <SettingsView
:tab="activeTab"
:theme="themeMode"
@theme-change="themeMode = normalizeTheme($event)"
/>
</section> </section>
</main> </main>
<Toast v-if="toast" :kind="toast.kind" :text="toast.text" /> <Toast v-if="toast" :kind="toast.kind" :text="toast.text" />
<div
v-if="operationStatus"
class="operation-hud"
:class="operationStatus.state"
>
<span v-if="operationStatus.state === 'running'" class="spinner" />
<span v-else class="status-mark">
{{ operationStatus.state === 'success' ? 'OK' : '!' }}
</span>
<div>
<strong>{{ operationStatus.phase }}</strong>
<p>{{ operationStatus.message }}</p>
</div>
</div>
</div> </div>
</template> </template>
@@ -277,6 +396,15 @@ onUnmounted(() => {
color: #111827; color: #111827;
font-size: 13px; font-size: 13px;
} }
.app-root.theme-light {
color-scheme: light;
}
.app-root.theme-dark {
color-scheme: dark;
}
.app-root.theme-auto {
color-scheme: light dark;
}
.status-pill { .status-pill {
font-size: 11px; font-size: 11px;
@@ -371,33 +499,124 @@ onUnmounted(() => {
min-width: 0; min-width: 0;
} }
.operation-hud {
position: fixed;
right: 18px;
bottom: 18px;
z-index: 20;
display: flex;
align-items: center;
gap: 12px;
min-width: 260px;
max-width: 360px;
padding: 12px 14px;
color: #f9fafb;
background: rgba(17, 24, 39, 0.94);
border-radius: 8px;
box-shadow: 0 12px 34px rgba(15, 23, 42, 0.28);
}
.operation-hud strong {
display: block;
margin-bottom: 2px;
font-size: 13px;
}
.operation-hud p {
margin: 0;
color: #d1d5db;
font-size: 12px;
line-height: 1.35;
}
.spinner {
width: 20px;
height: 20px;
border: 2px solid rgba(255, 255, 255, 0.25);
border-top-color: #60a5fa;
border-radius: 50%;
animation: spin 0.75s linear infinite;
flex: 0 0 auto;
}
.status-mark {
display: grid;
place-items: center;
width: 24px;
height: 24px;
border-radius: 50%;
font-size: 11px;
font-weight: 700;
flex: 0 0 auto;
}
.operation-hud.success .status-mark {
color: #052e16;
background: #86efac;
}
.operation-hud.error .status-mark {
color: #450a0a;
background: #fca5a5;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
.app-root.theme-dark {
background: #1c1d22;
color: #e5e7eb;
}
.app-root.theme-dark .titlebar {
background: rgba(28, 29, 34, 0.7);
border-bottom-color: #2c2f36;
}
.app-root.theme-dark .titlebar-title {
color: #f3f4f6;
}
.app-root.theme-dark .sidebar {
background: rgba(0, 0, 0, 0.15);
border-right-color: #2c2f36;
}
.app-root.theme-dark .sidebar-item {
color: #d1d5db;
}
.app-root.theme-dark .sidebar-item:hover:not(.active) {
background: rgba(255, 255, 255, 0.05);
}
.app-root.theme-dark .sidebar-item.active {
background: rgba(59, 130, 246, 0.18);
color: #93c5fd;
}
.app-root.theme-dark .permission-banner {
background: rgba(120, 53, 15, 0.3);
border-color: rgba(253, 186, 116, 0.4);
color: #fed7aa;
}
@media (prefers-color-scheme: dark) { @media (prefers-color-scheme: dark) {
.app-root { .app-root.theme-auto {
background: #1c1d22; background: #1c1d22;
color: #e5e7eb; color: #e5e7eb;
} }
.titlebar { .app-root.theme-auto .titlebar {
background: rgba(28, 29, 34, 0.7); background: rgba(28, 29, 34, 0.7);
border-bottom-color: #2c2f36; border-bottom-color: #2c2f36;
} }
.titlebar-title { .app-root.theme-auto .titlebar-title {
color: #f3f4f6; color: #f3f4f6;
} }
.sidebar { .app-root.theme-auto .sidebar {
background: rgba(0, 0, 0, 0.15); background: rgba(0, 0, 0, 0.15);
border-right-color: #2c2f36; border-right-color: #2c2f36;
} }
.sidebar-item { .app-root.theme-auto .sidebar-item {
color: #d1d5db; color: #d1d5db;
} }
.sidebar-item:hover:not(.active) { .app-root.theme-auto .sidebar-item:hover:not(.active) {
background: rgba(255, 255, 255, 0.05); background: rgba(255, 255, 255, 0.05);
} }
.sidebar-item.active { .app-root.theme-auto .sidebar-item.active {
background: rgba(59, 130, 246, 0.18); background: rgba(59, 130, 246, 0.18);
color: #93c5fd; color: #93c5fd;
} }
.permission-banner { .app-root.theme-auto .permission-banner {
background: rgba(120, 53, 15, 0.3); background: rgba(120, 53, 15, 0.3);
border-color: rgba(253, 186, 116, 0.4); border-color: rgba(253, 186, 116, 0.4);
color: #fed7aa; color: #fed7aa;
+186 -10
View File
@@ -1,5 +1,5 @@
<script setup lang="ts"> <script setup lang="ts">
import { computed, onMounted, onUnmounted, ref } from 'vue' import { computed, nextTick, onMounted, onUnmounted, ref } from 'vue'
// Inline SVG markup imported as raw strings via Vite's `?raw` suffix. // Inline SVG markup imported as raw strings via Vite's `?raw` suffix.
// Rationale: rendering through v-html lets the icon inherit `currentColor` // Rationale: rendering through v-html lets the icon inherit `currentColor`
// from the toolbar button, so styling stays in CSS without bundling extra // from the toolbar button, so styling stays in CSS without bundling extra
@@ -23,7 +23,7 @@ interface Rect {
h: number h: number
} }
type Tool = 'pen' | 'rect' | 'ellipse' type Tool = 'pen' | 'rect' | 'ellipse' | 'text'
interface Point { interface Point {
x: number x: number
y: number y: number
@@ -32,6 +32,7 @@ interface Annotation {
tool: Tool tool: Tool
color: string color: string
points: Point[] points: Point[]
text?: string
} }
const emit = defineEmits<{ const emit = defineEmits<{
@@ -51,6 +52,14 @@ const emit = defineEmits<{
e: 'save-remote', e: 'save-remote',
payload: { rect: Rect; annotations: Annotation[] } payload: { rect: Rect; annotations: Annotation[] }
): void ): void
(
e: 'summarize',
payload: { rect: Rect; annotations: Annotation[] }
): void
(
e: 'ocr',
payload: { rect: Rect; annotations: Annotation[] }
): void
(e: 'cancel'): void (e: 'cancel'): void
}>() }>()
@@ -60,6 +69,8 @@ const draftAnnotation = ref<Annotation | null>(null)
const activeTool = ref<Tool>('pen') const activeTool = ref<Tool>('pen')
const activeColor = ref('#ef4444') const activeColor = ref('#ef4444')
const paletteOpen = ref(false) const paletteOpen = ref(false)
const textDraft = ref<{ point: Point; value: string } | null>(null)
const textInputRef = ref<HTMLInputElement | null>(null)
type ResizeHandle = 'n' | 's' | 'e' | 'w' | 'nw' | 'ne' | 'sw' | 'se' type ResizeHandle = 'n' | 's' | 'e' | 'w' | 'nw' | 'ne' | 'sw' | 'se'
type DragMode = 'idle' | 'creating' | 'moving' | 'resizing' | 'annotating' type DragMode = 'idle' | 'creating' | 'moving' | 'resizing' | 'annotating'
@@ -111,14 +122,24 @@ const sizeLabel = computed(() => {
return `${Math.round(rect.value.w)} × ${Math.round(rect.value.h)}` return `${Math.round(rect.value.w)} × ${Math.round(rect.value.h)}`
}) })
const MARK_TOOLBAR_W = 222
const ACTION_TOOLBAR_W = 252
const TOOLBAR_GROUP_GAP = 8
const rightToolbarPos = computed(() => { const rightToolbarPos = computed(() => {
if (!rect.value) return null if (!rect.value) return null
return placeToolbar(rect.value, 180, 40, 'right') return placeToolbar(rect.value, ACTION_TOOLBAR_W, 40, 'right')
}) })
const leftToolbarPos = computed(() => { const leftToolbarPos = computed(() => {
if (!rect.value) return null if (!rect.value || !rightToolbarPos.value) return null
return placeToolbar(rect.value, 190, 40, 'left') // 标记组与操作组合并到右侧,紧贴在操作组左侧,中间保留间隔
const x = clamp(
rightToolbarPos.value.x - TOOLBAR_GROUP_GAP - MARK_TOOLBAR_W,
0,
props.width - MARK_TOOLBAR_W
)
return { x, y: rightToolbarPos.value.y }
}) })
const handles = computed(() => { const handles = computed(() => {
@@ -213,6 +234,7 @@ function hitHandle(p: Point): ResizeHandle | null {
function onMouseDown(e: MouseEvent) { function onMouseDown(e: MouseEvent) {
if (e.button !== 0) return if (e.button !== 0) return
commitTextDraft()
paletteOpen.value = false paletteOpen.value = false
const p = pointFromEvent(e) const p = pointFromEvent(e)
const handle = hitHandle(p) const handle = hitHandle(p)
@@ -239,6 +261,10 @@ function onSelectionMouseDown(e: MouseEvent) {
if (e.button !== 0 || !rect.value) return if (e.button !== 0 || !rect.value) return
e.stopPropagation() e.stopPropagation()
paletteOpen.value = false paletteOpen.value = false
if (activeTool.value === 'text') {
beginTextAnnotation(localPoint(pointFromEvent(e)))
return
}
if (e.detail >= 2) { if (e.detail >= 2) {
removeSinglePointAnnotation() removeSinglePointAnnotation()
onCopy() onCopy()
@@ -353,8 +379,17 @@ function onSaveRemote() {
emitAction('save-remote') emitAction('save-remote')
} }
function emitAction(action: 'confirm' | 'copy' | 'save' | 'save-remote') { function onSummarize() {
emitAction('summarize')
}
function onOCR() {
emitAction('ocr')
}
function emitAction(action: 'confirm' | 'copy' | 'save' | 'save-remote' | 'summarize' | 'ocr') {
if (!rect.value) return if (!rect.value) return
commitTextDraft()
const payload = { const payload = {
rect: { ...rect.value }, rect: { ...rect.value },
annotations: annotations.value, annotations: annotations.value,
@@ -363,6 +398,8 @@ function emitAction(action: 'confirm' | 'copy' | 'save' | 'save-remote') {
if (action === 'copy') emit('copy', payload) if (action === 'copy') emit('copy', payload)
if (action === 'save') emit('save', payload) if (action === 'save') emit('save', payload)
if (action === 'save-remote') emit('save-remote', payload) if (action === 'save-remote') emit('save-remote', payload)
if (action === 'summarize') emit('summarize', payload)
if (action === 'ocr') emit('ocr', payload)
} }
function onCancel() { function onCancel() {
@@ -370,6 +407,13 @@ function onCancel() {
} }
function onKeydown(e: KeyboardEvent) { function onKeydown(e: KeyboardEvent) {
if (textDraft.value) {
if (e.key === 'Escape') {
e.preventDefault()
cancelTextDraft()
}
return
}
if (e.key === 'Escape') { if (e.key === 'Escape') {
e.preventDefault() e.preventDefault()
onCancel() onCancel()
@@ -380,6 +424,7 @@ function onKeydown(e: KeyboardEvent) {
} }
function undoAnnotation() { function undoAnnotation() {
cancelTextDraft()
annotations.value.pop() annotations.value.pop()
} }
@@ -390,6 +435,45 @@ function removeSinglePointAnnotation() {
} }
} }
function beginTextAnnotation(point: Point) {
commitTextDraft()
dragMode.value = 'idle'
draftAnnotation.value = null
textDraft.value = { point, value: '' }
void nextTick(() => {
textInputRef.value?.focus()
textInputRef.value?.select()
})
}
function commitTextDraft() {
if (!textDraft.value) return
const text = textDraft.value.value.trim()
if (text !== '') {
annotations.value.push({
tool: 'text',
color: activeColor.value,
points: [textDraft.value.point],
text,
})
}
textDraft.value = null
}
function cancelTextDraft() {
textDraft.value = null
}
function onTextDraftKeydown(e: KeyboardEvent) {
if (e.key === 'Enter') {
e.preventDefault()
commitTextDraft()
} else if (e.key === 'Escape') {
e.preventDefault()
cancelTextDraft()
}
}
onMounted(() => { onMounted(() => {
window.addEventListener('keydown', onKeydown) window.addEventListener('keydown', onKeydown)
}) })
@@ -456,6 +540,17 @@ onUnmounted(() => {
stroke-width="3" stroke-width="3"
fill="none" fill="none"
/> />
<text
v-else-if="annotation.tool === 'text' && annotation.points.length >= 1"
:x="annotation.points[0].x"
:y="annotation.points[0].y"
:fill="annotation.color"
font-size="20"
font-weight="600"
dominant-baseline="hanging"
>
{{ annotation.text }}
</text>
</g> </g>
</svg> </svg>
@@ -471,6 +566,22 @@ onUnmounted(() => {
@mousedown="onSelectionMouseDown" @mousedown="onSelectionMouseDown"
/> />
<input
v-if="rect && textDraft"
ref="textInputRef"
v-model="textDraft.value"
class="text-editor"
:style="{
left: rect.x + textDraft.point.x + 'px',
top: rect.y + textDraft.point.y + 'px',
color: activeColor,
}"
spellcheck="false"
@mousedown.stop
@keydown.stop="onTextDraftKeydown"
@blur="commitTextDraft"
/>
<div <div
v-for="handle in handles" v-for="handle in handles"
:key="handle.name" :key="handle.name"
@@ -535,6 +646,18 @@ onUnmounted(() => {
<circle cx="12" cy="12" r="7" /> <circle cx="12" cy="12" r="7" />
</svg> </svg>
</button> </button>
<button
class="icon-btn"
:class="{ active: activeTool === 'text' }"
title="文字标记"
@click="selectTool('text')"
>
<svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M5 5h14" />
<path d="M12 5v14" />
<path d="M9 19h6" />
</svg>
</button>
<div class="color-wrap"> <div class="color-wrap">
<button <button
class="icon-btn color-btn" class="icon-btn color-btn"
@@ -583,8 +706,8 @@ onUnmounted(() => {
/> />
<button <button
class="action-btn" class="action-btn"
data-tip="复制图" data-tip="复制图"
aria-label="复制图" aria-label="复制图"
@click="onCopy" @click="onCopy"
v-html="copyIcon" v-html="copyIcon"
/> />
@@ -602,6 +725,34 @@ onUnmounted(() => {
@click="onSaveRemote" @click="onSaveRemote"
v-html="saveRemoteIcon" v-html="saveRemoteIcon"
/> />
<button
class="action-btn"
data-tip="提取文字"
aria-label="提取文字"
@click="onOCR"
>
<svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M5 8V5h3" />
<path d="M16 5h3v3" />
<path d="M19 16v3h-3" />
<path d="M8 19H5v-3" />
<path d="M9 15l3-7 3 7" />
<path d="M10 13h4" />
</svg>
</button>
<button
class="action-btn"
data-tip="复制总结"
aria-label="复制总结"
@click="onSummarize"
>
<svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M6 3h8l4 4v14H6z" />
<path d="M14 3v5h4" />
<path d="M9 12h6" />
<path d="M9 16h5" />
</svg>
</button>
<button <button
class="action-btn primary" class="action-btn primary"
data-tip="上传云端" data-tip="上传云端"
@@ -639,6 +790,21 @@ onUnmounted(() => {
cursor: crosshair; cursor: crosshair;
} }
.text-editor {
position: absolute;
z-index: 6;
min-width: 120px;
max-width: 320px;
height: 28px;
padding: 2px 6px;
border: 1px solid currentColor;
border-radius: 4px;
outline: none;
background: rgba(15, 23, 42, 0.72);
box-shadow: 0 4px 18px rgba(0, 0, 0, 0.35);
font: 600 20px/1.2 -apple-system, BlinkMacSystemFont, "SF Pro Text", sans-serif;
}
.resize-handle { .resize-handle {
position: absolute; position: absolute;
width: 10px; width: 10px;
@@ -693,10 +859,10 @@ onUnmounted(() => {
} }
.mark-toolbar { .mark-toolbar {
width: 190px; width: 222px;
} }
.action-toolbar { .action-toolbar {
width: 180px; width: 252px;
} }
.icon-btn, .icon-btn,
@@ -752,6 +918,16 @@ onUnmounted(() => {
fill: currentColor; fill: currentColor;
pointer-events: none; pointer-events: none;
} }
.action-btn > svg {
width: 18px;
height: 18px;
fill: none;
stroke: currentColor;
stroke-width: 2;
stroke-linecap: round;
stroke-linejoin: round;
pointer-events: none;
}
.action-btn::after { .action-btn::after {
content: attr(data-tip); content: attr(data-tip);
position: absolute; position: absolute;
+485 -30
View File
@@ -8,10 +8,12 @@
* • "general" — the global hotkey / capture parameters. * • "general" — the global hotkey / capture parameters.
* • "s3" — S3-compatible object-storage credentials. * • "s3" — S3-compatible object-storage credentials.
* • "ssh" — SSH/SCP destination for the "save remote" button. * • "ssh" — SSH/SCP destination for the "save remote" button.
* • "llm" — multimodal screenshot summary provider settings.
* • "ocr" — cloud OCR provider settings for text extraction.
* *
* State flow: load() pulls config from Go on mount, save() pushes back. * State flow: load() pulls config from Go on mount, save() pushes back.
*/ */
import { computed, onMounted, ref } from 'vue' import { computed, onMounted, ref, watch } from 'vue'
import { import {
GetConfig, GetConfig,
SaveConfig, SaveConfig,
@@ -22,10 +24,16 @@ import {
// Tab discriminator shared with the App shell. Kept as a string union so // Tab discriminator shared with the App shell. Kept as a string union so
// the parent can pass the value without importing a type from this view. // the parent can pass the value without importing a type from this view.
type TabId = 'general' | 's3' | 'ssh' type TabId = 'general' | 's3' | 'ssh' | 'llm' | 'ocr'
type ThemeMode = 'auto' | 'light' | 'dark'
const props = defineProps<{ const props = defineProps<{
tab: TabId tab: TabId
theme: ThemeMode
}>()
const emit = defineEmits<{
(event: 'theme-change', theme: ThemeMode): void
}>() }>()
interface S3Config { interface S3Config {
@@ -51,12 +59,74 @@ interface SSHConfig {
connectTimeoutSecs: number connectTimeoutSecs: number
} }
interface LLMProviderConfig {
label: string
baseUrl: string
apiKey: string
model: string
maxTokens: number
temperature: number
timeoutSecs: number
}
interface LLMConfig {
activeProvider: string
prompt: string
providers: Record<string, LLMProviderConfig>
}
interface OCRProviderConfig {
label: string
endpoint: string
accessKeyId: string
accessKeySecret: string
region: string
service: string
action: string
version: string
timeoutSecs: number
}
interface OCRConfig {
activeProvider: string
providers: Record<string, OCRProviderConfig>
}
interface AppConfig { interface AppConfig {
hotkey: string hotkey: string
theme: ThemeMode
s3: S3Config s3: S3Config
ssh: SSHConfig ssh: SSHConfig
llm: LLMConfig
ocr: OCRConfig
} }
const llmProviderOptions = [
{ id: 'qwen', label: '阿里千问多模态' },
{ id: 'doubao', label: '火山方舟豆包多模态' },
{ id: 'openai', label: 'ChatGPT / OpenAI-compatible' },
]
const ocrProviderOptions = [
{ id: 'aliyun', label: '阿里云文字识别' },
{ id: 'volcengine', label: '火山引擎文字识别' },
]
const themeOptions: Array<{ id: ThemeMode; label: string }> = [
{ id: 'light', label: '浅色' },
{ id: 'dark', label: '深色' },
{ id: 'auto', label: '自动' },
]
const DEFAULT_SUMMARY_PROMPT = `你是一个截图内容总结助手。请阅读截图,并用中文输出一段适合直接粘贴到聊天、工单、Issue 或 PR 中的说明。
要求:
- 先说明截图中最重要的信息和用户可能想表达的意图。
- 如果截图包含报错、异常状态、表格、代码、界面控件或关键数字,请准确提取。
- 如果截图内容不足以判断,不要编造;直接说明不确定点。
- 输出尽量简洁,默认 3 到 6 条要点。
- 不要输出 Markdown 标题,不要寒暄。`
// `defaultConfig` keeps the initial render in sync with the Go-side // `defaultConfig` keeps the initial render in sync with the Go-side
// DefaultAppConfig so the UI never flashes empty fields before load() // DefaultAppConfig so the UI never flashes empty fields before load()
// completes. Centralising the defaults also avoids subtle drift between // completes. Centralising the defaults also avoids subtle drift between
@@ -64,6 +134,7 @@ interface AppConfig {
function defaultConfig(): AppConfig { function defaultConfig(): AppConfig {
return { return {
hotkey: 'cmd+shift+a', hotkey: 'cmd+shift+a',
theme: 'auto',
s3: { s3: {
endpoint: '', endpoint: '',
region: 'us-east-1', region: 'us-east-1',
@@ -85,6 +156,66 @@ function defaultConfig(): AppConfig {
knownHostsPath: '', knownHostsPath: '',
connectTimeoutSecs: 10, connectTimeoutSecs: 10,
}, },
llm: {
activeProvider: 'openai',
prompt: DEFAULT_SUMMARY_PROMPT,
providers: {
qwen: {
label: '阿里千问多模态',
baseUrl: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
apiKey: '',
model: 'qwen-vl-plus',
maxTokens: 600,
temperature: 0.2,
timeoutSecs: 60,
},
doubao: {
label: '火山方舟豆包多模态',
baseUrl: 'https://ark.cn-beijing.volces.com/api/v3',
apiKey: '',
model: '',
maxTokens: 600,
temperature: 0.2,
timeoutSecs: 60,
},
openai: {
label: 'ChatGPT / OpenAI-compatible',
baseUrl: 'https://api.openai.com/v1',
apiKey: '',
model: 'gpt-5.5',
maxTokens: 600,
temperature: 0.2,
timeoutSecs: 60,
},
},
},
ocr: {
activeProvider: 'aliyun',
providers: {
aliyun: {
label: '阿里云文字识别',
endpoint: 'https://ocr-api.cn-hangzhou.aliyuncs.com',
accessKeyId: '',
accessKeySecret: '',
region: '',
service: '',
action: 'RecognizeGeneral',
version: '2021-07-07',
timeoutSecs: 30,
},
volcengine: {
label: '火山引擎文字识别',
endpoint: 'https://visual.volcengineapi.com',
accessKeyId: '',
accessKeySecret: '',
region: 'cn-north-1',
service: 'cv',
action: 'OCRNormal',
version: '2020-08-26',
timeoutSecs: 30,
},
},
},
} }
} }
@@ -109,6 +240,30 @@ const sshPathDisplay = computed({
}, },
}) })
const activeLLMProvider = computed(() => {
const id = config.value.llm.activeProvider || 'openai'
if (!config.value.llm.providers[id]) {
config.value.llm.activeProvider = 'openai'
return config.value.llm.providers.openai
}
return config.value.llm.providers[id]
})
const activeOCRProvider = computed(() => {
const id = config.value.ocr.activeProvider || 'aliyun'
if (!config.value.ocr.providers[id]) {
config.value.ocr.activeProvider = 'aliyun'
return config.value.ocr.providers.aliyun
}
return config.value.ocr.providers[id]
})
function normalizeTheme(theme: unknown): ThemeMode {
return theme === 'light' || theme === 'dark' || theme === 'auto'
? theme
: 'auto'
}
function flash(kind: 'ok' | 'err', text: string) { function flash(kind: 'ok' | 'err', text: string) {
message.value = { kind, text } message.value = { kind, text }
window.setTimeout(() => { window.setTimeout(() => {
@@ -125,7 +280,19 @@ async function load() {
Object.assign(merged, cfg) Object.assign(merged, cfg)
merged.s3 = { ...merged.s3, ...(cfg as any).s3 } merged.s3 = { ...merged.s3, ...(cfg as any).s3 }
merged.ssh = { ...merged.ssh, ...(cfg as any).ssh } merged.ssh = { ...merged.ssh, ...(cfg as any).ssh }
merged.llm = { ...merged.llm, ...(cfg as any).llm }
merged.llm.providers = {
...defaultConfig().llm.providers,
...((cfg as any).llm?.providers ?? {}),
}
merged.ocr = { ...merged.ocr, ...(cfg as any).ocr }
merged.ocr.providers = {
...defaultConfig().ocr.providers,
...((cfg as any).ocr?.providers ?? {}),
}
merged.theme = normalizeTheme((cfg as any).theme)
config.value = merged config.value = merged
emit('theme-change', config.value.theme)
} }
} }
@@ -169,11 +336,16 @@ async function manualCapture() {
await CaptureNow() await CaptureNow()
} }
watch(
() => config.value.theme,
(theme) => emit('theme-change', normalizeTheme(theme))
)
onMounted(load) onMounted(load)
</script> </script>
<template> <template>
<div class="settings"> <div class="settings" :class="`theme-${props.theme}`">
<header class="hero"> <header class="hero">
<div> <div>
<h1>SnapGo</h1> <h1>SnapGo</h1>
@@ -185,17 +357,31 @@ onMounted(load)
<button class="btn primary" @click="manualCapture">Capture now</button> <button class="btn primary" @click="manualCapture">Capture now</button>
</header> </header>
<!-- General tab only the shortcut configuration lives here. --> <!-- General tab shortcut and appearance configuration. -->
<section v-if="props.tab === 'general'" class="card"> <section v-if="props.tab === 'general'" class="card">
<h2>Shortcut</h2> <h2>General</h2>
<label class="field"> <div class="grid">
<span>Global hotkey</span> <label class="field">
<input <span>Global hotkey</span>
v-model="config.hotkey" <input
placeholder="cmd+shift+a" v-model="config.hotkey"
spellcheck="false" placeholder="cmd+shift+a"
/> spellcheck="false"
</label> />
</label>
<label class="field">
<span>主题选择</span>
<select v-model="config.theme">
<option
v-for="themeOption in themeOptions"
:key="themeOption.id"
:value="themeOption.id"
>
{{ themeOption.label }}
</option>
</select>
</label>
</div>
<p class="hint"> <p class="hint">
Tokens (case-insensitive, "+"-separated): cmd / ctrl / option / shift + Tokens (case-insensitive, "+"-separated): cmd / ctrl / option / shift +
az or 09. Example: <code>cmd+shift+a</code>. az or 09. Example: <code>cmd+shift+a</code>.
@@ -368,6 +554,200 @@ onMounted(load)
</div> </div>
</section> </section>
<!-- LLM tab multimodal screenshot summary provider settings. -->
<section v-if="props.tab === 'llm'" class="card">
<h2>LLM screenshot summary</h2>
<div class="grid">
<label class="field full">
<span>Provider</span>
<select v-model="config.llm.activeProvider">
<option
v-for="provider in llmProviderOptions"
:key="provider.id"
:value="provider.id"
>
{{ provider.label }}
</option>
</select>
</label>
<label class="field full">
<span>Base URL *</span>
<input
v-model="activeLLMProvider.baseUrl"
placeholder="https://api.openai.com/v1"
spellcheck="false"
/>
</label>
<label class="field">
<span>API Key *</span>
<input v-model="activeLLMProvider.apiKey" type="password" />
</label>
<label class="field">
<span>Model *</span>
<input
v-model="activeLLMProvider.model"
placeholder="gpt-5.5 / qwen-vl-plus / Ark endpoint ID"
spellcheck="false"
/>
</label>
<label class="field">
<span>Max tokens</span>
<input
v-model.number="activeLLMProvider.maxTokens"
type="number"
min="64"
max="4096"
placeholder="600"
/>
</label>
<label class="field">
<span>Temperature</span>
<input
v-model.number="activeLLMProvider.temperature"
type="number"
min="0"
max="2"
step="0.1"
placeholder="0.2"
/>
</label>
<label class="field">
<span>Timeout (sec)</span>
<input
v-model.number="activeLLMProvider.timeoutSecs"
type="number"
min="10"
max="300"
placeholder="60"
/>
</label>
<label class="field full">
<span>Prompt</span>
<textarea
v-model="config.llm.prompt"
class="prompt-textarea"
spellcheck="false"
/>
</label>
</div>
<p class="hint">
The "copy summary" screenshot action first uploads the image through the
S3 configuration, then sends the public image URL to this multimodal
provider. Keep S3 configured before using it.
</p>
<p v-if="config.llm.activeProvider === 'doubao'" class="hint">
Volcengine Ark usually uses the inference endpoint ID as the model
value; paste the endpoint/model identifier from your Ark console.
</p>
<p v-else-if="config.llm.activeProvider === 'qwen'" class="hint">
DashScope OpenAI-compatible mode is used here. If your workspace uses a
regional endpoint, replace Base URL with that compatible-mode URL.
</p>
<div class="actions">
<button class="btn primary" :disabled="saving" @click="save">
{{ saving ? 'Saving' : 'Save' }}
</button>
</div>
</section>
<!-- OCR tab cloud OCR provider settings for screenshot text extraction. -->
<section v-if="props.tab === 'ocr'" class="card">
<h2>文字提取</h2>
<div class="grid">
<label class="field full">
<span>Provider</span>
<select v-model="config.ocr.activeProvider">
<option
v-for="provider in ocrProviderOptions"
:key="provider.id"
:value="provider.id"
>
{{ provider.label }}
</option>
</select>
</label>
<label class="field full">
<span>Endpoint *</span>
<input
v-model="activeOCRProvider.endpoint"
placeholder="https://ocr-api.cn-hangzhou.aliyuncs.com"
spellcheck="false"
/>
</label>
<label class="field">
<span>AccessKey ID *</span>
<input v-model="activeOCRProvider.accessKeyId" spellcheck="false" />
</label>
<label class="field">
<span>AccessKey Secret *</span>
<input v-model="activeOCRProvider.accessKeySecret" type="password" />
</label>
<label
v-if="config.ocr.activeProvider === 'volcengine'"
class="field"
>
<span>Region *</span>
<input
v-model="activeOCRProvider.region"
placeholder="cn-north-1"
spellcheck="false"
/>
</label>
<label
v-if="config.ocr.activeProvider === 'volcengine'"
class="field"
>
<span>Service *</span>
<input
v-model="activeOCRProvider.service"
placeholder="cv"
spellcheck="false"
/>
</label>
<label class="field">
<span>Action *</span>
<input
v-model="activeOCRProvider.action"
placeholder="RecognizeGeneral / OCRNormal"
spellcheck="false"
/>
</label>
<label class="field">
<span>Version *</span>
<input
v-model="activeOCRProvider.version"
placeholder="2021-07-07 / 2020-08-26"
spellcheck="false"
/>
</label>
<label class="field">
<span>Timeout (sec)</span>
<input
v-model.number="activeOCRProvider.timeoutSecs"
type="number"
min="5"
max="120"
placeholder="30"
/>
</label>
</div>
<p v-if="config.ocr.activeProvider === 'aliyun'" class="hint">
阿里云默认使用 OCR API <code>RecognizeGeneral</code>请求体为截图
PNG 原始数据并使用 ACS3-HMAC-SHA256 签名
</p>
<p v-else class="hint">
火山引擎默认使用视觉智能的 <code>OCRNormal</code>截图以
<code>image_base64</code> 发送并使用 Region/Service 参与签名
</p>
<div class="actions">
<button class="btn primary" :disabled="saving" @click="save">
{{ saving ? 'Saving' : 'Save' }}
</button>
</div>
</section>
<transition name="fade"> <transition name="fade">
<div <div
v-if="message" v-if="message"
@@ -387,6 +767,15 @@ onMounted(load)
padding: 28px 32px 60px; padding: 28px 32px 60px;
color: #1f2937; color: #1f2937;
} }
.settings.theme-light {
color-scheme: light;
}
.settings.theme-dark {
color-scheme: dark;
}
.settings.theme-auto {
color-scheme: light dark;
}
.hero { .hero {
display: flex; display: flex;
align-items: center; align-items: center;
@@ -460,13 +849,13 @@ kbd {
.field input[type='text'], .field input[type='text'],
.field input[type='number'], .field input[type='number'],
.field input:not([type='checkbox']), .field input:not([type='checkbox']),
.field select { .field select,
.field textarea {
border: 1px solid #d1d5db; border: 1px solid #d1d5db;
border-radius: 6px; border-radius: 6px;
padding: 7px 10px; padding: 7px 10px;
font-size: 13px; font-size: 13px;
line-height: 1.4; line-height: 1.4;
height: 36px;
background: #fff; background: #fff;
color: inherit; color: inherit;
outline: none; outline: none;
@@ -474,8 +863,20 @@ kbd {
width: 100%; width: 100%;
box-sizing: border-box; box-sizing: border-box;
} }
.field input[type='text'],
.field input[type='number'],
.field input:not([type='checkbox']),
.field select {
height: 36px;
}
.field textarea {
min-height: 168px;
resize: vertical;
font-family: inherit;
}
.field input:focus, .field input:focus,
.field select:focus { .field select:focus,
.field textarea:focus {
border-color: #3b82f6; border-color: #3b82f6;
box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.18); box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.18);
} }
@@ -590,46 +991,100 @@ kbd {
.fade-leave-to { .fade-leave-to {
opacity: 0; opacity: 0;
} }
.settings.theme-dark {
color: #e5e7eb;
}
.settings.theme-dark .card {
background: #1f2937;
border-color: #374151;
}
.settings.theme-dark .field span {
color: #d1d5db;
}
.settings.theme-dark .field input[type='text'],
.settings.theme-dark .field input[type='number'],
.settings.theme-dark .field input:not([type='checkbox']),
.settings.theme-dark .field select,
.settings.theme-dark .field textarea {
background: #111827;
border-color: #374151;
color: #fff;
}
.settings.theme-dark .field input::placeholder,
.settings.theme-dark .field textarea::placeholder {
color: #9ca3af;
}
.settings.theme-dark .field input:disabled {
background: #1f2937;
color: #6b7280;
}
.settings.theme-dark .prefixed-input {
background: #111827;
border-color: #374151;
}
.settings.theme-dark .input-prefix {
background: #1f2937;
border-right-color: #374151;
color: #d1d5db;
}
.settings.theme-dark .btn:not(.primary) {
background: #1f2937;
border-color: #374151;
color: #e5e7eb;
}
.settings.theme-dark .btn:not(.primary):hover:not(:disabled) {
background: #374151;
}
.settings.theme-dark .subtitle {
color: #9ca3af;
}
@media (prefers-color-scheme: dark) { @media (prefers-color-scheme: dark) {
.settings { .settings.theme-auto {
color: #e5e7eb; color: #e5e7eb;
} }
.card { .settings.theme-auto .card {
background: #1f2937; background: #1f2937;
border-color: #374151; border-color: #374151;
} }
.field input { .settings.theme-auto .field span {
color: #d1d5db;
}
.settings.theme-auto .field input[type='text'],
.settings.theme-auto .field input[type='number'],
.settings.theme-auto .field input:not([type='checkbox']),
.settings.theme-auto .field select,
.settings.theme-auto .field textarea {
background: #111827; background: #111827;
border-color: #374151; border-color: #374151;
color: #e5e7eb; color: #fff;
} }
.field select { .settings.theme-auto .field input::placeholder,
background: #111827; .settings.theme-auto .field textarea::placeholder {
border-color: #374151; color: #9ca3af;
color: #e5e7eb;
} }
.field input:disabled { .settings.theme-auto .field input:disabled {
background: #1f2937; background: #1f2937;
color: #6b7280; color: #6b7280;
} }
.prefixed-input { .settings.theme-auto .prefixed-input {
background: #111827; background: #111827;
border-color: #374151; border-color: #374151;
} }
.input-prefix { .settings.theme-auto .input-prefix {
background: #1f2937; background: #1f2937;
border-right-color: #374151; border-right-color: #374151;
color: #d1d5db; color: #d1d5db;
} }
.btn { .settings.theme-auto .btn:not(.primary) {
background: #1f2937; background: #1f2937;
border-color: #374151; border-color: #374151;
color: #e5e7eb; color: #e5e7eb;
} }
.btn:hover:not(:disabled) { .settings.theme-auto .btn:not(.primary):hover:not(:disabled) {
background: #374151; background: #374151;
} }
.subtitle { .settings.theme-auto .subtitle {
color: #9ca3af; color: #9ca3af;
} }
} }
+8
View File
@@ -17,6 +17,10 @@ export function CopyNativeRegionImage(arg1:main.CaptureResult):Promise<void>;
export function CopyRegionImage(arg1:main.CaptureResult):Promise<void>; export function CopyRegionImage(arg1:main.CaptureResult):Promise<void>;
export function ExtractTextNativeRegion(arg1:main.CaptureResult):Promise<void>;
export function ExtractTextRegion(arg1:main.CaptureResult):Promise<void>;
export function GetConfig():Promise<domain.AppConfig>; export function GetConfig():Promise<domain.AppConfig>;
export function QuitApp():Promise<void>; export function QuitApp():Promise<void>;
@@ -37,6 +41,10 @@ export function SaveRegionToRemote(arg1:main.CaptureResult):Promise<void>;
export function ShowWindow():Promise<void>; export function ShowWindow():Promise<void>;
export function SummarizeNativeRegion(arg1:main.CaptureResult):Promise<void>;
export function SummarizeRegion(arg1:main.CaptureResult):Promise<void>;
export function TestConnection(arg1:domain.S3Config):Promise<void>; export function TestConnection(arg1:domain.S3Config):Promise<void>;
export function TestSSHConnection(arg1:domain.SSHConfig):Promise<void>; export function TestSSHConnection(arg1:domain.SSHConfig):Promise<void>;
+16
View File
@@ -30,6 +30,14 @@ export function CopyRegionImage(arg1) {
return window['go']['main']['App']['CopyRegionImage'](arg1); return window['go']['main']['App']['CopyRegionImage'](arg1);
} }
export function ExtractTextNativeRegion(arg1) {
return window['go']['main']['App']['ExtractTextNativeRegion'](arg1);
}
export function ExtractTextRegion(arg1) {
return window['go']['main']['App']['ExtractTextRegion'](arg1);
}
export function GetConfig() { export function GetConfig() {
return window['go']['main']['App']['GetConfig'](); return window['go']['main']['App']['GetConfig']();
} }
@@ -70,6 +78,14 @@ export function ShowWindow() {
return window['go']['main']['App']['ShowWindow'](); return window['go']['main']['App']['ShowWindow']();
} }
export function SummarizeNativeRegion(arg1) {
return window['go']['main']['App']['SummarizeNativeRegion'](arg1);
}
export function SummarizeRegion(arg1) {
return window['go']['main']['App']['SummarizeRegion'](arg1);
}
export function TestConnection(arg1) { export function TestConnection(arg1) {
return window['go']['main']['App']['TestConnection'](arg1); return window['go']['main']['App']['TestConnection'](arg1);
} }
+132
View File
@@ -18,6 +18,7 @@ export namespace application {
tool: string; tool: string;
color: string; color: string;
points: Point[]; points: Point[];
text?: string;
static createFrom(source: any = {}) { static createFrom(source: any = {}) {
return new Annotation(source); return new Annotation(source);
@@ -28,6 +29,7 @@ export namespace application {
this.tool = source["tool"]; this.tool = source["tool"];
this.color = source["color"]; this.color = source["color"];
this.points = this.convertValues(source["points"], Point); this.points = this.convertValues(source["points"], Point);
this.text = source["text"];
} }
convertValues(a: any, classs: any, asMap: boolean = false): any { convertValues(a: any, classs: any, asMap: boolean = false): any {
@@ -53,6 +55,126 @@ export namespace application {
export namespace domain { export namespace domain {
export class OCRProviderConfig {
label: string;
endpoint: string;
accessKeyId: string;
accessKeySecret: string;
region: string;
service: string;
action: string;
version: string;
timeoutSecs: number;
static createFrom(source: any = {}) {
return new OCRProviderConfig(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.label = source["label"];
this.endpoint = source["endpoint"];
this.accessKeyId = source["accessKeyId"];
this.accessKeySecret = source["accessKeySecret"];
this.region = source["region"];
this.service = source["service"];
this.action = source["action"];
this.version = source["version"];
this.timeoutSecs = source["timeoutSecs"];
}
}
export class OCRConfig {
activeProvider: string;
providers: Record<string, OCRProviderConfig>;
static createFrom(source: any = {}) {
return new OCRConfig(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.activeProvider = source["activeProvider"];
this.providers = this.convertValues(source["providers"], OCRProviderConfig, true);
}
convertValues(a: any, classs: any, asMap: boolean = false): any {
if (!a) {
return a;
}
if (a.slice && a.map) {
return (a as any[]).map(elem => this.convertValues(elem, classs));
} else if ("object" === typeof a) {
if (asMap) {
for (const key of Object.keys(a)) {
a[key] = new classs(a[key]);
}
return a;
}
return new classs(a);
}
return a;
}
}
export class LLMProviderConfig {
label: string;
baseUrl: string;
apiKey: string;
model: string;
maxTokens: number;
temperature: number;
timeoutSecs: number;
maxInlineBytes: number;
static createFrom(source: any = {}) {
return new LLMProviderConfig(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.label = source["label"];
this.baseUrl = source["baseUrl"];
this.apiKey = source["apiKey"];
this.model = source["model"];
this.maxTokens = source["maxTokens"];
this.temperature = source["temperature"];
this.timeoutSecs = source["timeoutSecs"];
this.maxInlineBytes = source["maxInlineBytes"];
}
}
export class LLMConfig {
activeProvider: string;
prompt: string;
providers: Record<string, LLMProviderConfig>;
static createFrom(source: any = {}) {
return new LLMConfig(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.activeProvider = source["activeProvider"];
this.prompt = source["prompt"];
this.providers = this.convertValues(source["providers"], LLMProviderConfig, true);
}
convertValues(a: any, classs: any, asMap: boolean = false): any {
if (!a) {
return a;
}
if (a.slice && a.map) {
return (a as any[]).map(elem => this.convertValues(elem, classs));
} else if ("object" === typeof a) {
if (asMap) {
for (const key of Object.keys(a)) {
a[key] = new classs(a[key]);
}
return a;
}
return new classs(a);
}
return a;
}
}
export class SSHConfig { export class SSHConfig {
host: string; host: string;
port: number; port: number;
@@ -109,8 +231,11 @@ export namespace domain {
} }
export class AppConfig { export class AppConfig {
hotkey: string; hotkey: string;
theme: string;
s3: S3Config; s3: S3Config;
ssh: SSHConfig; ssh: SSHConfig;
llm: LLMConfig;
ocr: OCRConfig;
static createFrom(source: any = {}) { static createFrom(source: any = {}) {
return new AppConfig(source); return new AppConfig(source);
@@ -119,8 +244,11 @@ export namespace domain {
constructor(source: any = {}) { constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source); if ('string' === typeof source) source = JSON.parse(source);
this.hotkey = source["hotkey"]; this.hotkey = source["hotkey"];
this.theme = source["theme"];
this.s3 = this.convertValues(source["s3"], S3Config); this.s3 = this.convertValues(source["s3"], S3Config);
this.ssh = this.convertValues(source["ssh"], SSHConfig); this.ssh = this.convertValues(source["ssh"], SSHConfig);
this.llm = this.convertValues(source["llm"], LLMConfig);
this.ocr = this.convertValues(source["ocr"], OCRConfig);
} }
convertValues(a: any, classs: any, asMap: boolean = false): any { convertValues(a: any, classs: any, asMap: boolean = false): any {
@@ -143,6 +271,10 @@ export namespace domain {
} }
} }
export namespace main { export namespace main {
+1 -1
View File
@@ -12,6 +12,7 @@ require (
golang.design/x/clipboard v0.7.0 golang.design/x/clipboard v0.7.0
golang.design/x/hotkey v0.4.1 golang.design/x/hotkey v0.4.1
golang.org/x/crypto v0.33.0 golang.org/x/crypto v0.33.0
golang.org/x/image v0.12.0
) )
require ( require (
@@ -52,7 +53,6 @@ require (
github.com/wailsapp/go-webview2 v1.0.22 // indirect github.com/wailsapp/go-webview2 v1.0.22 // indirect
github.com/wailsapp/mimetype v1.4.1 // indirect github.com/wailsapp/mimetype v1.4.1 // indirect
golang.org/x/exp v0.0.0-20190731235908-ec7cb31e5a56 // indirect golang.org/x/exp v0.0.0-20190731235908-ec7cb31e5a56 // indirect
golang.org/x/image v0.12.0 // indirect
golang.org/x/mobile v0.0.0-20230301163155-e0f57694e12c // indirect golang.org/x/mobile v0.0.0-20230301163155-e0f57694e12c // indirect
golang.org/x/net v0.35.0 // indirect golang.org/x/net v0.35.0 // indirect
golang.org/x/sys v0.30.0 // indirect golang.org/x/sys v0.30.0 // indirect
+97
View File
@@ -8,8 +8,15 @@ import (
"image/draw" "image/draw"
"image/png" "image/png"
"math" "math"
"os"
"strconv" "strconv"
"strings" "strings"
"sync"
xfont "golang.org/x/image/font"
"golang.org/x/image/font/basicfont"
"golang.org/x/image/font/opentype"
"golang.org/x/image/math/fixed"
) )
// Annotation describes a user-drawn mark relative to the selected screenshot. // Annotation describes a user-drawn mark relative to the selected screenshot.
@@ -18,6 +25,7 @@ type Annotation struct {
Tool string `json:"tool"` Tool string `json:"tool"`
Color string `json:"color"` Color string `json:"color"`
Points []Point `json:"points"` Points []Point `json:"points"`
Text string `json:"text,omitempty"`
} }
type Point struct { type Point struct {
@@ -55,6 +63,8 @@ func ApplyAnnotations(pngBytes []byte, annotations []Annotation, scale float64)
drawRectOutline(dst, ann.Points, scale, width, c) drawRectOutline(dst, ann.Points, scale, width, c)
case "ellipse": case "ellipse":
drawEllipseOutline(dst, ann.Points, scale, width, c) drawEllipseOutline(dst, ann.Points, scale, width, c)
case "text":
drawTextAnnotation(dst, ann, scale, c)
} }
} }
@@ -180,3 +190,90 @@ func ordered(a, b float64) (float64, float64) {
} }
return b, a return b, a
} }
func drawTextAnnotation(img *image.RGBA, ann Annotation, scale float64, c color.RGBA) {
if len(ann.Points) == 0 {
return
}
text := strings.TrimSpace(ann.Text)
if text == "" {
return
}
face := annotationFontFace(20 * scale)
if face == nil {
face = basicfont.Face7x13
}
origin := scalePoint(ann.Points[0], scale)
metrics := face.Metrics()
lineHeight := metrics.Height
if lineHeight <= 0 {
lineHeight = fixed.I(int(math.Ceil(24 * scale)))
}
d := &xfont.Drawer{
Dst: img,
Src: image.NewUniform(c),
Face: face,
}
baselineY := fixed.I(int(math.Round(origin.Y))) + metrics.Ascent
x := fixed.I(int(math.Round(origin.X)))
for _, line := range strings.Split(text, "\n") {
line = strings.TrimRight(line, "\r")
if line != "" {
d.Dot = fixed.Point26_6{X: x, Y: baselineY}
d.DrawString(line)
}
baselineY += lineHeight
}
}
var (
annotationFontOnce sync.Once
annotationFont *opentype.Font
)
func annotationFontFace(size float64) xfont.Face {
if size <= 0 {
size = 20
}
annotationFontOnce.Do(func() {
annotationFont = loadAnnotationFont()
})
if annotationFont == nil {
return basicfont.Face7x13
}
face, err := opentype.NewFace(annotationFont, &opentype.FaceOptions{
Size: size,
DPI: 72,
Hinting: xfont.HintingFull,
})
if err != nil {
return basicfont.Face7x13
}
return face
}
func loadAnnotationFont() *opentype.Font {
paths := []string{
"/System/Library/Fonts/Supplemental/Arial Unicode.ttf",
"/Library/Fonts/Arial Unicode.ttf",
"/System/Library/Fonts/Hiragino Sans GB.ttc",
"/System/Library/Fonts/PingFang.ttc",
"/System/Library/Fonts/Helvetica.ttc",
}
for _, path := range paths {
data, err := os.ReadFile(path)
if err != nil {
continue
}
if collection, err := opentype.ParseCollection(data); err == nil && collection.NumFonts() > 0 {
if font, err := collection.Font(0); err == nil {
return font
}
}
if font, err := opentype.Parse(data); err == nil {
return font
}
}
return nil
}
+42
View File
@@ -51,3 +51,45 @@ func TestApplyAnnotationsNoAnnotationsReturnsOriginalBytes(t *testing.T) {
t.Fatalf("expected original bytes") t.Fatalf("expected original bytes")
} }
} }
func TestApplyAnnotationsDrawsTextIntoPNG(t *testing.T) {
src := image.NewRGBA(image.Rect(0, 0, 120, 80))
draw.Draw(src, src.Bounds(), &image.Uniform{C: color.White}, image.Point{}, draw.Src)
var buf bytes.Buffer
if err := png.Encode(&buf, src); err != nil {
t.Fatalf("encode source: %v", err)
}
out, err := ApplyAnnotations(buf.Bytes(), []Annotation{
{
Tool: "text",
Color: "#ef4444",
Points: []Point{
{X: 10, Y: 10},
},
Text: "T",
},
}, 1)
if err != nil {
t.Fatalf("apply annotations: %v", err)
}
img, err := png.Decode(bytes.NewReader(out))
if err != nil {
t.Fatalf("decode output: %v", err)
}
found := false
for y := 8; y < 40 && !found; y++ {
for x := 8; x < 40; x++ {
got := color.RGBAModel.Convert(img.At(x, y)).(color.RGBA)
if got.R > 180 && got.G < 180 && got.B < 180 {
found = true
break
}
}
}
if !found {
t.Fatalf("expected red text pixels near annotation point")
}
}
+8
View File
@@ -61,6 +61,14 @@ func (s *CaptureActionsService) SaveImage(_ context.Context, pngBytes []byte, di
s.notifyFailure("save failed: " + err.Error()) s.notifyFailure("save failed: " + err.Error())
return "", err return "", err
} }
// Copy the saved path to the clipboard so the user can paste it right
// away, mirroring how the upload/summary flows copy their result. A
// clipboard failure must not fail the save itself.
if s.Clipboard != nil {
if err := s.Clipboard.WriteText(path); err != nil {
s.notifyFailure("clipboard write failed: " + err.Error())
}
}
if s.Notifier != nil { if s.Notifier != nil {
s.Notifier.NotifySuccess(path) s.Notifier.NotifySuccess(path)
} }
+57
View File
@@ -0,0 +1,57 @@
package application
import (
"context"
"fmt"
"strings"
"github.com/mmmy/snapgo/internal/infrastructure/clipboard"
)
// OCRRecognizer describes a provider capable of extracting text from a PNG
// screenshot.
type OCRRecognizer interface {
RecognizeText(ctx context.Context, pngBytes []byte) (string, error)
}
// CaptureOCRService wires screenshot bytes -> OCR provider -> clipboard. It
// leaves progress reporting to the caller so native and web overlays can share
// the same status HUD.
type CaptureOCRService struct {
Recognizer OCRRecognizer
Clipboard clipboard.Writer
}
// Recognize asks the configured OCR provider to extract text from the PNG.
func (s *CaptureOCRService) Recognize(ctx context.Context, pngBytes []byte) (string, error) {
if s.Recognizer == nil {
return "", fmt.Errorf("ocr is not configured")
}
if len(pngBytes) == 0 {
return "", fmt.Errorf("empty screenshot")
}
text, err := s.Recognizer.RecognizeText(ctx, pngBytes)
if err != nil {
return "", err
}
text = strings.TrimSpace(text)
if text == "" {
return "", fmt.Errorf("ocr result is empty")
}
return text, nil
}
// CopyText writes the extracted text to the clipboard.
func (s *CaptureOCRService) CopyText(_ context.Context, text string) error {
text = strings.TrimSpace(text)
if text == "" {
return fmt.Errorf("empty ocr text")
}
if s.Clipboard == nil {
return fmt.Errorf("clipboard is not configured")
}
if err := s.Clipboard.WriteText(text); err != nil {
return fmt.Errorf("clipboard write failed: %w", err)
}
return nil
}
+50
View File
@@ -0,0 +1,50 @@
package application
import (
"context"
"testing"
)
type fakeOCRRecognizer struct {
image []byte
text string
}
func (f *fakeOCRRecognizer) RecognizeText(_ context.Context, pngBytes []byte) (string, error) {
f.image = append([]byte(nil), pngBytes...)
if f.text == "" {
return "extracted text", nil
}
return f.text, nil
}
func TestCaptureOCRServiceRecognizesAndCopies(t *testing.T) {
ctx := context.Background()
recognizer := &fakeOCRRecognizer{}
clip := &fakeClipboard{}
svc := &CaptureOCRService{Recognizer: recognizer, Clipboard: clip}
text, err := svc.Recognize(ctx, []byte("png"))
if err != nil {
t.Fatalf("recognize: %v", err)
}
if string(recognizer.image) != "png" {
t.Fatalf("expected screenshot bytes forwarded, got %q", string(recognizer.image))
}
if text != "extracted text" {
t.Fatalf("expected extracted text, got %q", text)
}
if err := svc.CopyText(ctx, text); err != nil {
t.Fatalf("copy text: %v", err)
}
if clip.text != "extracted text" {
t.Fatalf("expected ocr text copied, got %q", clip.text)
}
}
func TestCaptureOCRServiceRejectsEmptyResult(t *testing.T) {
svc := &CaptureOCRService{Recognizer: &fakeOCRRecognizer{text: " \n "}}
if _, err := svc.Recognize(context.Background(), []byte("png")); err == nil {
t.Fatalf("expected empty OCR result to fail")
}
}
+158
View File
@@ -0,0 +1,158 @@
package application
import (
"context"
"encoding/base64"
"fmt"
"net/http"
"time"
"github.com/mmmy/snapgo/internal/domain"
"github.com/mmmy/snapgo/internal/infrastructure/clipboard"
)
// VisionSummarizer describes a multimodal model capable of reading an image
// reference (either a public https URL or an inline base64 data URL) and
// returning a textual summary.
type VisionSummarizer interface {
SummarizeImage(ctx context.Context, prompt string, imageURL string) (string, error)
}
// ObjectDeleter is an optional capability an OSSProvider may implement so the
// summary pipeline can delete the temporary screenshot it uploaded purely so a
// remote model could fetch it. It is kept separate from domain.OSSProvider to
// avoid forcing every provider (or test fake) to implement deletion.
type ObjectDeleter interface {
Delete(ctx context.Context, key string) error
}
// CaptureSummaryService wires screenshot bytes -> multimodal LLM summary ->
// clipboard. It prefers sending the image inline as a base64 data URL and only
// falls back to an S3 upload (public URL) when the payload exceeds the
// provider's inline size limit. It intentionally does not notify UI state
// itself: the caller emits fine-grained "uploading / recognizing / done"
// progress.
type CaptureSummaryService struct {
Provider domain.OSSProvider
Summarizer VisionSummarizer
Clipboard clipboard.Writer
Prompt string
PathPrefix string
// MaxInlineBytes caps the PNG size eligible for inline base64. Payloads
// larger than this must go through S3. A value <= 0 disables the inline
// path entirely (always upload).
MaxInlineBytes int
// HTTPClient is used for the public-URL reachability probe. Defaults to a
// short-timeout client when nil.
HTTPClient *http.Client
}
// CanInline reports whether the screenshot is small enough to be embedded
// directly in the request as a base64 data URL.
func (s *CaptureSummaryService) CanInline(pngBytes []byte) bool {
return s.MaxInlineBytes > 0 && len(pngBytes) <= s.MaxInlineBytes
}
// InlineDataURL encodes the PNG as an RFC 2397 data URL suitable for the
// image_url field of an OpenAI-compatible request.
func (s *CaptureSummaryService) InlineDataURL(pngBytes []byte) string {
return "data:image/png;base64," + base64.StdEncoding.EncodeToString(pngBytes)
}
// UploadImage uploads the screenshot and returns the public URL visible to
// the multimodal provider. Used only when the image is too large to inline.
func (s *CaptureSummaryService) UploadImage(ctx context.Context, pngBytes []byte) (*domain.UploadResult, error) {
if s.Provider == nil {
return nil, fmt.Errorf("s3 is not configured")
}
if len(pngBytes) == 0 {
return nil, fmt.Errorf("empty screenshot")
}
key := buildObjectKey(s.PathPrefix)
start := time.Now()
url, err := s.Provider.Upload(ctx, key, pngBytes, "image/png")
if err != nil {
return nil, fmt.Errorf("s3 put object: %w", err)
}
return &domain.UploadResult{
URL: url,
Key: key,
Provider: s.Provider.Name(),
Elapsed: time.Since(start),
}, nil
}
// EnsureReachable verifies the public URL can actually be fetched, so we can
// surface a clear "link not reachable" message instead of an opaque provider
// error when the bucket is private or fronted by an inaccessible CDN.
func (s *CaptureSummaryService) EnsureReachable(ctx context.Context, imageURL string) error {
client := s.HTTPClient
if client == nil {
client = &http.Client{Timeout: 10 * time.Second}
}
probe := func(method string) (int, error) {
req, err := http.NewRequestWithContext(ctx, method, imageURL, nil)
if err != nil {
return 0, err
}
resp, err := client.Do(req)
if err != nil {
return 0, err
}
resp.Body.Close()
return resp.StatusCode, nil
}
status, err := probe(http.MethodHead)
// Some S3-compatible endpoints reject HEAD; retry with GET before failing.
if err != nil || status == http.StatusMethodNotAllowed || status == http.StatusForbidden {
if s, gerr := probe(http.MethodGet); gerr == nil {
status, err = s, nil
} else if err == nil {
err = gerr
}
}
if err != nil {
return fmt.Errorf("图片链接不可达(模型无法读取上传的截图):%w", err)
}
if status < 200 || status >= 300 {
return fmt.Errorf("图片链接不可达:HTTP %d,请确认对象存储 bucket 公网可读或已正确配置 PublicURLBase", status)
}
return nil
}
// DeleteUploaded removes the temporary object from S3 if the provider supports
// deletion. It is best-effort: a nil return means either success or that the
// provider cannot delete.
func (s *CaptureSummaryService) DeleteUploaded(ctx context.Context, key string) error {
if key == "" {
return nil
}
deleter, ok := s.Provider.(ObjectDeleter)
if !ok {
return nil
}
return deleter.Delete(ctx, key)
}
// Summarize asks the configured model to summarize the image reference, which
// may be a public URL or an inline base64 data URL.
func (s *CaptureSummaryService) Summarize(ctx context.Context, imageURL string) (string, error) {
if s.Summarizer == nil {
return "", fmt.Errorf("llm is not configured")
}
return s.Summarizer.SummarizeImage(ctx, s.Prompt, imageURL)
}
// CopySummary writes the final summary text to the clipboard.
func (s *CaptureSummaryService) CopySummary(_ context.Context, summary string) error {
if summary == "" {
return fmt.Errorf("empty summary")
}
if s.Clipboard == nil {
return fmt.Errorf("clipboard is not configured")
}
if err := s.Clipboard.WriteText(summary); err != nil {
return fmt.Errorf("clipboard write failed: %w", err)
}
return nil
}
@@ -0,0 +1,140 @@
package application
import (
"context"
"strings"
"testing"
)
type fakeOSSProvider struct {
key string
data []byte
contentType string
url string
deletedKey string
}
func (f *fakeOSSProvider) Upload(_ context.Context, key string, data []byte, contentType string) (string, error) {
f.key = key
f.data = append([]byte(nil), data...)
f.contentType = contentType
if f.url == "" {
return "https://cdn.example.com/" + key, nil
}
return f.url, nil
}
func (f *fakeOSSProvider) Name() string { return "fake" }
func (f *fakeOSSProvider) Delete(_ context.Context, key string) error {
f.deletedKey = key
return nil
}
type fakeSummarizer struct {
prompt string
url string
text string
}
func (f *fakeSummarizer) SummarizeImage(_ context.Context, prompt string, imageURL string) (string, error) {
f.prompt = prompt
f.url = imageURL
if f.text == "" {
return "summary text", nil
}
return f.text, nil
}
func TestCaptureSummaryServiceUploadsSummarizesAndCopies(t *testing.T) {
ctx := context.Background()
provider := &fakeOSSProvider{}
summarizer := &fakeSummarizer{}
clip := &fakeClipboard{}
svc := &CaptureSummaryService{
Provider: provider,
Summarizer: summarizer,
Clipboard: clip,
Prompt: "describe screenshot",
PathPrefix: "snapgo/",
}
uploaded, err := svc.UploadImage(ctx, []byte("png"))
if err != nil {
t.Fatalf("upload image: %v", err)
}
if provider.contentType != "image/png" {
t.Fatalf("expected image/png content type, got %q", provider.contentType)
}
if string(provider.data) != "png" {
t.Fatalf("expected uploaded png bytes, got %q", string(provider.data))
}
summary, err := svc.Summarize(ctx, uploaded.URL)
if err != nil {
t.Fatalf("summarize: %v", err)
}
if summarizer.prompt != "describe screenshot" {
t.Fatalf("expected prompt forwarded, got %q", summarizer.prompt)
}
if summarizer.url != uploaded.URL {
t.Fatalf("expected uploaded url forwarded, got %q", summarizer.url)
}
if summary != "summary text" {
t.Fatalf("expected summary text, got %q", summary)
}
if err := svc.CopySummary(ctx, summary); err != nil {
t.Fatalf("copy summary: %v", err)
}
if clip.text != "summary text" {
t.Fatalf("expected summary copied, got %q", clip.text)
}
}
func TestCaptureSummaryServiceInlineDataURL(t *testing.T) {
svc := &CaptureSummaryService{MaxInlineBytes: 10}
if !svc.CanInline([]byte("small")) {
t.Fatalf("expected 5-byte payload to be inlinable under 10-byte cap")
}
if svc.CanInline([]byte("this is definitely too big")) {
t.Fatalf("expected oversized payload to be rejected for inlining")
}
url := svc.InlineDataURL([]byte("png"))
if !strings.HasPrefix(url, "data:image/png;base64,") {
t.Fatalf("expected base64 data URL prefix, got %q", url)
}
if strings.Contains(url, "https://") {
t.Fatalf("inline URL must not be a remote URL, got %q", url)
}
}
func TestCaptureSummaryServiceDeleteUploadedUsesDeleter(t *testing.T) {
provider := &fakeOSSProvider{}
svc := &CaptureSummaryService{Provider: provider}
if err := svc.DeleteUploaded(context.Background(), "snapgo/a.png"); err != nil {
t.Fatalf("delete uploaded: %v", err)
}
if provider.deletedKey != "snapgo/a.png" {
t.Fatalf("expected object deleted, got %q", provider.deletedKey)
}
}
func TestCaptureSummaryServiceDeleteUploadedNoDeleterIsNoop(t *testing.T) {
// A provider without Delete must not error when cleanup is requested.
svc := &CaptureSummaryService{Provider: nonDeletingProvider{}}
if err := svc.DeleteUploaded(context.Background(), "snapgo/a.png"); err != nil {
t.Fatalf("expected no-op delete, got %v", err)
}
}
type nonDeletingProvider struct{}
func (nonDeletingProvider) Upload(_ context.Context, _ string, _ []byte, _ string) (string, error) {
return "https://cdn.example.com/x.png", nil
}
func (nonDeletingProvider) Name() string { return "no-delete" }
+332 -18
View File
@@ -10,12 +10,12 @@ package domain
// endpoint, etc.). // endpoint, etc.).
// //
// Field design notes: // Field design notes:
// - PathPrefix supports object name templating so users can group screenshots // - PathPrefix supports object name templating so users can group screenshots
// by date. // by date.
// - PublicURLBase is optional to support CDN-fronted buckets; otherwise the // - PublicURLBase is optional to support CDN-fronted buckets; otherwise the
// adapter falls back to "{Endpoint}/{Bucket}/{Key}" (path-style). // adapter falls back to "{Endpoint}/{Bucket}/{Key}" (path-style).
// - UsePathStyle defaults to true because many self-hosted MinIO / R2 // - UsePathStyle defaults to true because many self-hosted MinIO / R2
// deployments do not support virtual-hosted-style addressing. // deployments do not support virtual-hosted-style addressing.
type S3Config struct { type S3Config struct {
Endpoint string `json:"endpoint"` Endpoint string `json:"endpoint"`
Region string `json:"region"` Region string `json:"region"`
@@ -34,17 +34,17 @@ type S3Config struct {
// - Host / Port form the destination endpoint. Port defaults to 22 when 0. // - Host / Port form the destination endpoint. Port defaults to 22 when 0.
// - User is the SSH login name. // - User is the SSH login name.
// - AuthMethod selects how the connection authenticates: // - AuthMethod selects how the connection authenticates:
// "password" → password only, via the in-process Go SSH client. // "password" → password only, via the in-process Go SSH client.
// "key" → ssh-agent + ~/.ssh/id_* key files (no password), // "key" → ssh-agent + ~/.ssh/id_* key files (no password),
// via the in-process Go SSH client. // via the in-process Go SSH client.
// "kerberos" → delegate to the system /usr/bin/ssh binary so the // "kerberos" → delegate to the system /usr/bin/ssh binary so the
// existing Kerberos (GSSAPI) credential cache from // existing Kerberos (GSSAPI) credential cache from
// `kinit` is reused. Native GSSAPI is required here // `kinit` is reused. Native GSSAPI is required here
// because macOS stores tickets in an API: ccache // because macOS stores tickets in an API: ccache
// that pure-Go SSH libraries cannot read. // that pure-Go SSH libraries cannot read.
// "" / "builtin" → legacy combined behaviour (password → agent → key // "" / "builtin" → legacy combined behaviour (password → agent → key
// files) kept only for backward compatibility with // files) kept only for backward compatibility with
// configs written before the method was split. // configs written before the method was split.
// - Password is optional; when empty the implementation falls back to the // - Password is optional; when empty the implementation falls back to the
// local SSH agent or ~/.ssh/id_* keys (i.e. the user is responsible for // local SSH agent or ~/.ssh/id_* keys (i.e. the user is responsible for
// having password-less access already configured on this machine). // having password-less access already configured on this machine).
@@ -72,6 +72,64 @@ type SSHConfig struct {
ConnectTimeoutSecs int `json:"connectTimeoutSecs"` ConnectTimeoutSecs int `json:"connectTimeoutSecs"`
} }
// LLMProviderConfig stores one OpenAI-compatible multimodal chat endpoint.
//
// The three built-in providers (Qwen, Doubao/Ark, OpenAI-compatible) all use
// the same request shape: chat/completions with a user content array that
// contains text plus an image_url item. Keeping them as provider records lets
// users switch between credentials/models without retyping every field.
type LLMProviderConfig struct {
Label string `json:"label"`
BaseURL string `json:"baseUrl"`
APIKey string `json:"apiKey"`
Model string `json:"model"`
MaxTokens int `json:"maxTokens"`
Temperature float64 `json:"temperature"`
TimeoutSecs int `json:"timeoutSecs"`
// MaxInlineBytes caps the screenshot size (in bytes of the PNG payload)
// that may be sent inline as a base64 data URL. Above this threshold the
// summary pipeline falls back to uploading the image to S3 and passing a
// public URL instead, because most providers reject oversized inline
// images. A value <= 0 means "use the built-in default".
MaxInlineBytes int `json:"maxInlineBytes"`
}
// DefaultMaxInlineBytes is the fallback inline-image cap (~4 MiB) used when a
// provider config does not specify MaxInlineBytes. It is a conservative bound
// that keeps a single base64 data URL within the request-size limits accepted
// by the common OpenAI-compatible multimodal endpoints.
const DefaultMaxInlineBytes = 4 << 20
// LLMConfig controls screenshot summarisation.
type LLMConfig struct {
ActiveProvider string `json:"activeProvider"`
Prompt string `json:"prompt"`
Providers map[string]LLMProviderConfig `json:"providers"`
}
// OCRProviderConfig stores one cloud OCR endpoint that accepts a screenshot
// image and returns extracted text. Aliyun and Volcengine both authenticate
// with an AccessKey pair, but their signing schemes use slightly different
// endpoint metadata; keeping Region/Service/Action/Version configurable lets
// the presets track official API changes without a schema rewrite.
type OCRProviderConfig struct {
Label string `json:"label"`
Endpoint string `json:"endpoint"`
AccessKeyID string `json:"accessKeyId"`
AccessKeySecret string `json:"accessKeySecret"`
Region string `json:"region"`
Service string `json:"service"`
Action string `json:"action"`
Version string `json:"version"`
TimeoutSecs int `json:"timeoutSecs"`
}
// OCRConfig controls screenshot text extraction.
type OCRConfig struct {
ActiveProvider string `json:"activeProvider"`
Providers map[string]OCRProviderConfig `json:"providers"`
}
// SSH authentication method identifiers stored in SSHConfig.AuthMethod. // SSH authentication method identifiers stored in SSHConfig.AuthMethod.
const ( const (
// SSHAuthBuiltin is the legacy combined method (password → agent → key // SSHAuthBuiltin is the legacy combined method (password → agent → key
@@ -87,6 +145,35 @@ const (
SSHAuthKerberos = "kerberos" SSHAuthKerberos = "kerberos"
) )
// Built-in LLM provider identifiers.
const (
LLMProviderQwen = "qwen"
LLMProviderDoubao = "doubao"
LLMProviderOpenAI = "openai"
)
// Built-in OCR provider identifiers.
const (
OCRProviderAliyun = "aliyun"
OCRProviderVolcengine = "volcengine"
)
// Theme preference identifiers stored in AppConfig.Theme.
const (
ThemeAuto = "auto"
ThemeLight = "light"
ThemeDark = "dark"
)
const DefaultSummaryPrompt = `你是一个截图内容总结助手请阅读截图并用中文输出一段适合直接粘贴到聊天工单Issue PR 中的说明
要求
- 先说明截图中最重要的信息和用户可能想表达的意图
- 如果截图包含报错异常状态表格代码界面控件或关键数字请准确提取
- 如果截图内容不足以判断不要编造直接说明不确定点
- 输出尽量简洁默认 3 6 条要点
- 不要输出 Markdown 标题不要寒暄`
// IsKerberos reports whether this config requests Kerberos/GSSAPI auth. // IsKerberos reports whether this config requests Kerberos/GSSAPI auth.
func (c SSHConfig) IsKerberos() bool { func (c SSHConfig) IsKerberos() bool {
return c.AuthMethod == SSHAuthKerberos return c.AuthMethod == SSHAuthKerberos
@@ -102,18 +189,28 @@ type AppConfig struct {
// in the infrastructure hotkey adapter. // in the infrastructure hotkey adapter.
Hotkey string `json:"hotkey"` Hotkey string `json:"hotkey"`
// Theme controls the settings UI appearance: "auto", "light", or "dark".
Theme string `json:"theme"`
// S3 holds the active S3-compatible storage configuration. // S3 holds the active S3-compatible storage configuration.
S3 S3Config `json:"s3"` S3 S3Config `json:"s3"`
// SSH holds the configuration for the optional "save to remote via scp" // SSH holds the configuration for the optional "save to remote via scp"
// destination triggered by the save-remote toolbar button. // destination triggered by the save-remote toolbar button.
SSH SSHConfig `json:"ssh"` SSH SSHConfig `json:"ssh"`
// LLM holds provider settings for the "copy summary" screenshot action.
LLM LLMConfig `json:"llm"`
// OCR holds provider settings for the "extract text" screenshot action.
OCR OCRConfig `json:"ocr"`
} }
// DefaultAppConfig returns sane zero-value defaults used on first launch. // DefaultAppConfig returns sane zero-value defaults used on first launch.
func DefaultAppConfig() AppConfig { func DefaultAppConfig() AppConfig {
return AppConfig{ cfg := AppConfig{
Hotkey: "cmd+shift+a", Hotkey: "cmd+shift+a",
Theme: ThemeAuto,
S3: S3Config{ S3: S3Config{
PathPrefix: "snapgo/", PathPrefix: "snapgo/",
UsePathStyle: true, UsePathStyle: true,
@@ -124,6 +221,176 @@ func DefaultAppConfig() AppConfig {
ConnectTimeoutSecs: 10, ConnectTimeoutSecs: 10,
StrictHostKey: false, StrictHostKey: false,
}, },
LLM: DefaultLLMConfig(),
OCR: DefaultOCRConfig(),
}
cfg.Normalize()
return cfg
}
// DefaultLLMConfig returns the built-in provider presets. API keys and exact
// models remain user-editable because each provider account may expose
// different model names / endpoint IDs.
func DefaultLLMConfig() LLMConfig {
return LLMConfig{
ActiveProvider: LLMProviderOpenAI,
Prompt: DefaultSummaryPrompt,
Providers: map[string]LLMProviderConfig{
LLMProviderQwen: {
Label: "阿里千问多模态",
BaseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
Model: "qwen-vl-plus",
MaxTokens: 600,
Temperature: 0.2,
TimeoutSecs: 60,
MaxInlineBytes: DefaultMaxInlineBytes,
},
LLMProviderDoubao: {
Label: "火山方舟豆包多模态",
BaseURL: "https://ark.cn-beijing.volces.com/api/v3",
Model: "",
MaxTokens: 600,
Temperature: 0.2,
TimeoutSecs: 60,
MaxInlineBytes: DefaultMaxInlineBytes,
},
LLMProviderOpenAI: {
Label: "ChatGPT / OpenAI-compatible",
BaseURL: "https://api.openai.com/v1",
Model: "gpt-5.5",
MaxTokens: 600,
Temperature: 0.2,
TimeoutSecs: 60,
MaxInlineBytes: DefaultMaxInlineBytes,
},
},
}
}
// DefaultOCRConfig returns the built-in OCR provider presets. Users only need
// to fill AccessKey credentials for the common regions/actions, while advanced
// accounts can still override endpoint metadata from the settings UI.
func DefaultOCRConfig() OCRConfig {
return OCRConfig{
ActiveProvider: OCRProviderAliyun,
Providers: map[string]OCRProviderConfig{
OCRProviderAliyun: {
Label: "阿里云文字识别",
Endpoint: "https://ocr-api.cn-hangzhou.aliyuncs.com",
Action: "RecognizeGeneral",
Version: "2021-07-07",
TimeoutSecs: 30,
},
OCRProviderVolcengine: {
Label: "火山引擎文字识别",
Endpoint: "https://visual.volcengineapi.com",
Region: "cn-north-1",
Service: "cv",
Action: "OCRNormal",
Version: "2020-08-26",
TimeoutSecs: 30,
},
},
}
}
// Normalize fills defaults into configs written by older app versions while
// preserving any user-provided provider fields.
func (c *AppConfig) Normalize() {
if c.Hotkey == "" {
c.Hotkey = "cmd+shift+a"
}
if c.Theme != ThemeLight && c.Theme != ThemeDark && c.Theme != ThemeAuto {
c.Theme = ThemeAuto
}
if c.S3.PathPrefix == "" {
c.S3.PathPrefix = "snapgo/"
}
if c.SSH.Port == 0 {
c.SSH.Port = 22
}
if c.SSH.PathPrefix == "" {
c.SSH.PathPrefix = "snapgo/"
}
if c.SSH.ConnectTimeoutSecs == 0 {
c.SSH.ConnectTimeoutSecs = 10
}
defaultLLM := DefaultLLMConfig()
if c.LLM.ActiveProvider == "" {
c.LLM.ActiveProvider = defaultLLM.ActiveProvider
}
if c.LLM.Prompt == "" {
c.LLM.Prompt = defaultLLM.Prompt
}
if c.LLM.Providers == nil {
c.LLM.Providers = map[string]LLMProviderConfig{}
}
for id, def := range defaultLLM.Providers {
current, ok := c.LLM.Providers[id]
if !ok {
c.LLM.Providers[id] = def
continue
}
if current.Label == "" {
current.Label = def.Label
}
if current.BaseURL == "" {
current.BaseURL = def.BaseURL
}
if current.Model == "" && id != LLMProviderDoubao {
current.Model = def.Model
}
if current.MaxTokens == 0 {
current.MaxTokens = def.MaxTokens
}
if current.Temperature < 0 {
current.Temperature = def.Temperature
}
if current.TimeoutSecs == 0 {
current.TimeoutSecs = def.TimeoutSecs
}
if current.MaxInlineBytes <= 0 {
current.MaxInlineBytes = DefaultMaxInlineBytes
}
c.LLM.Providers[id] = current
}
defaultOCR := DefaultOCRConfig()
if c.OCR.ActiveProvider == "" {
c.OCR.ActiveProvider = defaultOCR.ActiveProvider
}
if c.OCR.Providers == nil {
c.OCR.Providers = map[string]OCRProviderConfig{}
}
for id, def := range defaultOCR.Providers {
current, ok := c.OCR.Providers[id]
if !ok {
c.OCR.Providers[id] = def
continue
}
if current.Label == "" {
current.Label = def.Label
}
if current.Endpoint == "" {
current.Endpoint = def.Endpoint
}
if current.Region == "" {
current.Region = def.Region
}
if current.Service == "" {
current.Service = def.Service
}
if current.Action == "" {
current.Action = def.Action
}
if current.Version == "" {
current.Version = def.Version
}
if current.TimeoutSecs == 0 {
current.TimeoutSecs = def.TimeoutSecs
}
c.OCR.Providers[id] = current
} }
} }
@@ -141,3 +408,50 @@ func (c AppConfig) IsS3Configured() bool {
func (c AppConfig) IsSSHConfigured() bool { func (c AppConfig) IsSSHConfigured() bool {
return c.SSH.Host != "" && c.SSH.User != "" return c.SSH.Host != "" && c.SSH.User != ""
} }
// ActiveLLMProvider returns the selected provider config plus a boolean
// indicating whether the selection exists.
func (c AppConfig) ActiveLLMProvider() (string, LLMProviderConfig, bool) {
id := c.LLM.ActiveProvider
if id == "" {
id = LLMProviderOpenAI
}
provider, ok := c.LLM.Providers[id]
return id, provider, ok
}
// IsLLMConfigured reports whether the selected multimodal provider has the
// fields required to make a request.
func (c AppConfig) IsLLMConfigured() bool {
_, provider, ok := c.ActiveLLMProvider()
return ok && provider.BaseURL != "" && provider.APIKey != "" && provider.Model != ""
}
// ActiveOCRProvider returns the selected OCR provider config plus a boolean
// indicating whether the selection exists.
func (c AppConfig) ActiveOCRProvider() (string, OCRProviderConfig, bool) {
id := c.OCR.ActiveProvider
if id == "" {
id = OCRProviderAliyun
}
provider, ok := c.OCR.Providers[id]
return id, provider, ok
}
// IsOCRConfigured reports whether the selected OCR provider has the fields
// required to sign and send a request.
func (c AppConfig) IsOCRConfigured() bool {
id, provider, ok := c.ActiveOCRProvider()
if !ok ||
provider.Endpoint == "" ||
provider.AccessKeyID == "" ||
provider.AccessKeySecret == "" ||
provider.Action == "" ||
provider.Version == "" {
return false
}
if id == OCRProviderVolcengine && (provider.Region == "" || provider.Service == "") {
return false
}
return true
}
+8 -6
View File
@@ -1,12 +1,12 @@
// Package config provides JSON-file backed persistence for AppConfig. // Package config provides JSON-file backed persistence for AppConfig.
// //
// Design rationale: // Design rationale:
// - JSON keeps the file human-readable, which is useful while the MVP has // - JSON keeps the file human-readable, which is useful while the MVP has
// no settings UI for every option. // no settings UI for every option.
// - Storage location follows os.UserConfigDir() so each platform places it // - Storage location follows os.UserConfigDir() so each platform places it
// in the canonical spot (macOS: ~/Library/Application Support/SnapGo). // in the canonical spot (macOS: ~/Library/Application Support/SnapGo).
// - File mode 0600 / dir mode 0700 mitigate accidental leakage of access // - File mode 0600 / dir mode 0700 mitigate accidental leakage of access
// keys before we wire up the OS keychain (deferred to a later spec). // keys before we wire up the OS keychain (deferred to a later spec).
package config package config
import ( import (
@@ -65,6 +65,7 @@ func (s *FileStore) Load() (domain.AppConfig, error) {
if err := json.Unmarshal(data, &cfg); err != nil { if err := json.Unmarshal(data, &cfg); err != nil {
return domain.DefaultAppConfig(), fmt.Errorf("parse config: %w", err) return domain.DefaultAppConfig(), fmt.Errorf("parse config: %w", err)
} }
cfg.Normalize()
return cfg, nil return cfg, nil
} }
@@ -74,6 +75,7 @@ func (s *FileStore) Save(cfg domain.AppConfig) error {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
cfg.Normalize()
data, err := json.MarshalIndent(cfg, "", " ") data, err := json.MarshalIndent(cfg, "", " ")
if err != nil { if err != nil {
return fmt.Errorf("marshal config: %w", err) return fmt.Errorf("marshal config: %w", err)
+1 -1
View File
@@ -104,7 +104,7 @@ func parseSpec(spec string) ([]hk.Modifier, hk.Key, error) {
case "option", "alt": case "option", "alt":
mods = append(mods, modOption()) mods = append(mods, modOption())
case "shift": case "shift":
mods = append(mods, hk.ModShift) mods = append(mods, modShift())
default: default:
k, ok := lookupKey(p) k, ok := lookupKey(p)
if !ok { if !ok {
@@ -1,10 +1,15 @@
//go:build darwin //go:build darwin && cgo
package hotkey package hotkey
import hk "golang.design/x/hotkey" import hk "golang.design/x/hotkey"
// macOS uses ⌘ as the primary modifier; Option corresponds to Alt. // macOS uses ⌘ as the primary modifier; Option corresponds to Alt.
//
// This file is gated on cgo because golang.design/x/hotkey only exposes the
// Mod* constants from its cgo-backed darwin implementation; with CGO_ENABLED=0
// the package falls back to a stub that defines no constants.
func modCmd() hk.Modifier { return hk.ModCmd } func modCmd() hk.Modifier { return hk.ModCmd }
func modCtrl() hk.Modifier { return hk.ModCtrl } func modCtrl() hk.Modifier { return hk.ModCtrl }
func modOption() hk.Modifier { return hk.ModOption } func modOption() hk.Modifier { return hk.ModOption }
func modShift() hk.Modifier { return hk.ModShift }
@@ -0,0 +1,18 @@
//go:build linux && cgo
package hotkey
import hk "golang.design/x/hotkey"
// On Linux we map "cmd" to Ctrl so the same config string ("cmd+shift+a")
// stays meaningful across platforms.
//
// The X11 backend exposes modifiers as Mod1..Mod5 rather than named Alt/Option
// constants; Mod1 is the conventional Alt mask under X11, so "option"/"alt"
// resolve to hk.Mod1. This file is gated on cgo because the constants only
// exist in the cgo-backed linux implementation (CGO_ENABLED=0 falls back to a
// stub with no constants).
func modCmd() hk.Modifier { return hk.ModCtrl }
func modCtrl() hk.Modifier { return hk.ModCtrl }
func modOption() hk.Modifier { return hk.Mod1 }
func modShift() hk.Modifier { return hk.ModShift }
@@ -1,11 +0,0 @@
//go:build !darwin
package hotkey
import hk "golang.design/x/hotkey"
// On Windows / Linux we map "cmd" to Ctrl so the same config string works
// across platforms; "option" is treated as Alt.
func modCmd() hk.Modifier { return hk.ModCtrl }
func modCtrl() hk.Modifier { return hk.ModCtrl }
func modOption() hk.Modifier { return hk.ModAlt }
@@ -0,0 +1,25 @@
//go:build !windows && !cgo
package hotkey
import hk "golang.design/x/hotkey"
// This file covers the configuration where golang.design/x/hotkey compiles its
// cgo-less stub (any non-Windows platform built with CGO_ENABLED=0). In that
// stub the package defines the Modifier/Key types but no Mod*/Key* constants,
// and Register panics at runtime. We still provide the package-internal
// helpers so that snapgo keeps cross-compiling (e.g. `go vet`, editor analysis,
// or CI cross-builds) without pulling in a C toolchain.
//
// The returned values are inert zero modifiers and lookupKey always reports
// "unsupported": callers on this build cannot register a real global hotkey,
// so surfacing an "unsupported token" parse error is preferable to reaching
// the upstream panic.
func modCmd() hk.Modifier { return 0 }
func modCtrl() hk.Modifier { return 0 }
func modOption() hk.Modifier { return 0 }
func modShift() hk.Modifier { return 0 }
// lookupKey has no key constants to map against in the cgo-less stub, so it
// always reports the token as unsupported.
func lookupKey(string) (hk.Key, bool) { return 0, false }
@@ -0,0 +1,13 @@
//go:build windows
package hotkey
import hk "golang.design/x/hotkey"
// On Windows we map "cmd" to Ctrl so the same config string ("cmd+shift+a")
// stays meaningful across platforms; "option" is treated as Alt. The Windows
// backend does not require cgo, hence the single-tag guard.
func modCmd() hk.Modifier { return hk.ModCtrl }
func modCtrl() hk.Modifier { return hk.ModCtrl }
func modOption() hk.Modifier { return hk.ModAlt }
func modShift() hk.Modifier { return hk.ModShift }
+6
View File
@@ -1,9 +1,15 @@
//go:build windows || cgo
package hotkey package hotkey
import hk "golang.design/x/hotkey" import hk "golang.design/x/hotkey"
// lookupKey maps a token from the user-supplied hotkey spec to a hk.Key. // lookupKey maps a token from the user-supplied hotkey spec to a hk.Key.
// //
// The Key* constants only exist in the platform backends that ship them
// (windows, or the cgo-backed darwin/linux implementations); the cgo-less stub
// in hotkey_unsupported.go provides a fallback lookupKey for the complement.
//
// The map is intentionally exhaustive over the alphanumeric set so we never // The map is intentionally exhaustive over the alphanumeric set so we never
// rely on an assumption that hk.KeyA..hk.KeyZ are contiguous values — the // rely on an assumption that hk.KeyA..hk.KeyZ are contiguous values — the
// upstream package gives no such guarantee. // upstream package gives no such guarantee.
@@ -0,0 +1,174 @@
// Package llm contains multimodal LLM adapters.
//
// The built-in providers are intentionally implemented through the
// OpenAI-compatible chat/completions shape. Qwen DashScope, Volcengine Ark,
// and OpenAI-compatible gateways all accept the same "text + image_url"
// message content pattern, so one small HTTP adapter is enough here.
package llm
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/mmmy/snapgo/internal/domain"
)
// VisionClient calls one configured multimodal chat endpoint.
type VisionClient struct {
providerID string
cfg domain.LLMProviderConfig
httpClient *http.Client
}
// NewVisionClient validates cfg and returns a reusable client.
func NewVisionClient(providerID string, cfg domain.LLMProviderConfig) (*VisionClient, error) {
if strings.TrimSpace(cfg.BaseURL) == "" {
return nil, fmt.Errorf("llm base url is required")
}
if strings.TrimSpace(cfg.APIKey) == "" {
return nil, fmt.Errorf("llm api key is required")
}
if strings.TrimSpace(cfg.Model) == "" {
return nil, fmt.Errorf("llm model is required")
}
timeout := cfg.TimeoutSecs
if timeout <= 0 {
timeout = 60
}
return &VisionClient{
providerID: providerID,
cfg: cfg,
httpClient: &http.Client{Timeout: time.Duration(timeout) * time.Second},
}, nil
}
// SummarizeImage asks the model to summarise an image available at imageURL.
func (c *VisionClient) SummarizeImage(ctx context.Context, prompt string, imageURL string) (string, error) {
prompt = strings.TrimSpace(prompt)
if prompt == "" {
return "", fmt.Errorf("llm prompt is empty")
}
imageURL = strings.TrimSpace(imageURL)
if imageURL == "" {
return "", fmt.Errorf("image url is empty")
}
body := map[string]any{
"model": c.cfg.Model,
"messages": []map[string]any{
{
"role": "user",
"content": []map[string]any{
{"type": "text", "text": prompt},
{"type": "image_url", "image_url": map[string]any{"url": imageURL}},
},
},
},
}
if c.cfg.MaxTokens > 0 {
if c.providerID == domain.LLMProviderOpenAI {
body["max_completion_tokens"] = c.cfg.MaxTokens
} else {
body["max_tokens"] = c.cfg.MaxTokens
}
}
if c.cfg.Temperature >= 0 {
body["temperature"] = c.cfg.Temperature
}
payload, err := json.Marshal(body)
if err != nil {
return "", fmt.Errorf("marshal llm request: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, chatCompletionsURL(c.cfg.BaseURL), bytes.NewReader(payload))
if err != nil {
return "", fmt.Errorf("build llm request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+c.cfg.APIKey)
resp, err := c.httpClient.Do(req)
if err != nil {
return "", fmt.Errorf("llm request: %w", err)
}
defer resp.Body.Close()
data, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return "", fmt.Errorf("llm status %d: %s", resp.StatusCode, compactBody(data))
}
var decoded chatCompletionResponse
if err := json.Unmarshal(data, &decoded); err != nil {
return "", fmt.Errorf("decode llm response: %w", err)
}
if decoded.Error != nil && decoded.Error.Message != "" {
return "", fmt.Errorf("llm error: %s", decoded.Error.Message)
}
if len(decoded.Choices) == 0 {
return "", fmt.Errorf("llm response has no choices")
}
text := strings.TrimSpace(contentText(decoded.Choices[0].Message.Content))
if text == "" {
return "", fmt.Errorf("llm response is empty")
}
return text, nil
}
func chatCompletionsURL(baseURL string) string {
base := strings.TrimRight(strings.TrimSpace(baseURL), "/")
if strings.HasSuffix(base, "/chat/completions") {
return base
}
return base + "/chat/completions"
}
type chatCompletionResponse struct {
Choices []struct {
Message struct {
Content any `json:"content"`
} `json:"message"`
} `json:"choices"`
Error *struct {
Message string `json:"message"`
Type string `json:"type"`
Code any `json:"code"`
} `json:"error"`
}
func contentText(content any) string {
switch v := content.(type) {
case string:
return v
case []any:
parts := make([]string, 0, len(v))
for _, item := range v {
obj, ok := item.(map[string]any)
if !ok {
continue
}
if text, ok := obj["text"].(string); ok {
parts = append(parts, text)
}
}
return strings.Join(parts, "\n")
default:
return ""
}
}
func compactBody(data []byte) string {
text := strings.TrimSpace(string(data))
if text == "" {
return "<empty body>"
}
if len(text) > 800 {
return text[:800] + "..."
}
return text
}
@@ -0,0 +1,86 @@
package llm
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"testing"
"github.com/mmmy/snapgo/internal/domain"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func TestVisionClientSummarizeImageUsesChatCompletionImageURL(t *testing.T) {
var requestBody map[string]any
transport := roundTripFunc(func(r *http.Request) (*http.Response, error) {
if r.URL.Path != "/v1/chat/completions" {
t.Fatalf("expected /v1/chat/completions, got %s", r.URL.Path)
}
if got := r.Header.Get("Authorization"); got != "Bearer test-key" {
t.Fatalf("unexpected auth header %q", got)
}
if err := json.NewDecoder(r.Body).Decode(&requestBody); err != nil {
t.Fatalf("decode request: %v", err)
}
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(bytes.NewBufferString(`{"choices":[{"message":{"content":"summary ok"}}]}`)),
}, nil
})
client, err := NewVisionClient(domain.LLMProviderQwen, domain.LLMProviderConfig{
BaseURL: "https://example.test/v1",
APIKey: "test-key",
Model: "qwen-vl-plus",
MaxTokens: 321,
Temperature: 0.2,
TimeoutSecs: 5,
})
if err != nil {
t.Fatalf("new client: %v", err)
}
client.httpClient.Transport = transport
text, err := client.SummarizeImage(context.Background(), "describe", "https://cdn.example.com/a.png")
if err != nil {
t.Fatalf("summarize image: %v", err)
}
if text != "summary ok" {
t.Fatalf("expected response text, got %q", text)
}
if requestBody["model"] != "qwen-vl-plus" {
t.Fatalf("expected model forwarded, got %#v", requestBody["model"])
}
if requestBody["max_tokens"] != float64(321) {
t.Fatalf("expected max_tokens forwarded, got %#v", requestBody["max_tokens"])
}
messages := requestBody["messages"].([]any)
content := messages[0].(map[string]any)["content"].([]any)
if content[0].(map[string]any)["text"] != "describe" {
t.Fatalf("expected prompt content, got %#v", content[0])
}
image := content[1].(map[string]any)
if image["type"] != "image_url" {
t.Fatalf("expected image_url item, got %#v", image)
}
url := image["image_url"].(map[string]any)["url"]
if url != "https://cdn.example.com/a.png" {
t.Fatalf("expected image url forwarded, got %#v", url)
}
}
func TestChatCompletionsURLAcceptsFullEndpoint(t *testing.T) {
full := "https://example.com/v1/chat/completions"
if got := chatCompletionsURL(full); got != full {
t.Fatalf("expected full endpoint unchanged, got %q", got)
}
}
+568
View File
@@ -0,0 +1,568 @@
// Package ocr contains cloud OCR adapters.
package ocr
import (
"bytes"
"context"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"sort"
"strings"
"time"
"github.com/mmmy/snapgo/internal/application"
"github.com/mmmy/snapgo/internal/domain"
)
// Client calls one configured OCR provider.
type Client struct {
providerID string
cfg domain.OCRProviderConfig
httpClient *http.Client
now func() time.Time
nonce func() string
}
var _ application.OCRRecognizer = (*Client)(nil)
// NewClient validates cfg and returns a reusable OCR client.
func NewClient(providerID string, cfg domain.OCRProviderConfig) (*Client, error) {
if strings.TrimSpace(cfg.Endpoint) == "" {
return nil, fmt.Errorf("ocr endpoint is required")
}
if strings.TrimSpace(cfg.AccessKeyID) == "" {
return nil, fmt.Errorf("ocr access key id is required")
}
if strings.TrimSpace(cfg.AccessKeySecret) == "" {
return nil, fmt.Errorf("ocr access key secret is required")
}
if strings.TrimSpace(cfg.Action) == "" {
return nil, fmt.Errorf("ocr action is required")
}
if strings.TrimSpace(cfg.Version) == "" {
return nil, fmt.Errorf("ocr version is required")
}
switch providerID {
case domain.OCRProviderAliyun:
case domain.OCRProviderVolcengine:
if strings.TrimSpace(cfg.Region) == "" {
return nil, fmt.Errorf("volcengine ocr region is required")
}
if strings.TrimSpace(cfg.Service) == "" {
return nil, fmt.Errorf("volcengine ocr service is required")
}
default:
return nil, fmt.Errorf("unsupported ocr provider %q", providerID)
}
timeout := cfg.TimeoutSecs
if timeout <= 0 {
timeout = 30
}
return &Client{
providerID: providerID,
cfg: cfg,
httpClient: &http.Client{Timeout: time.Duration(timeout) * time.Second},
now: time.Now,
nonce: randomNonce,
}, nil
}
// RecognizeText extracts visible text from a PNG screenshot.
func (c *Client) RecognizeText(ctx context.Context, pngBytes []byte) (string, error) {
if len(pngBytes) == 0 {
return "", fmt.Errorf("empty screenshot")
}
switch c.providerID {
case domain.OCRProviderAliyun:
return c.recognizeAliyun(ctx, pngBytes)
case domain.OCRProviderVolcengine:
return c.recognizeVolcengine(ctx, pngBytes)
default:
return "", fmt.Errorf("unsupported ocr provider %q", c.providerID)
}
}
func (c *Client) recognizeAliyun(ctx context.Context, pngBytes []byte) (string, error) {
endpoint, err := parseEndpoint(c.cfg.Endpoint)
if err != nil {
return "", err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint.String(), bytes.NewReader(pngBytes))
if err != nil {
return "", fmt.Errorf("build aliyun ocr request: %w", err)
}
req.Header.Set("Accept", "application/json")
req.Header.Set("Content-Type", "application/octet-stream")
c.signAliyun(req, pngBytes)
return c.doOCRRequest(req, "aliyun ocr")
}
func (c *Client) recognizeVolcengine(ctx context.Context, pngBytes []byte) (string, error) {
endpoint, err := parseEndpoint(c.cfg.Endpoint)
if err != nil {
return "", err
}
query := endpoint.Query()
query.Set("Action", c.cfg.Action)
query.Set("Version", c.cfg.Version)
endpoint.RawQuery = query.Encode()
body := map[string]string{
"image_base64": base64.StdEncoding.EncodeToString(pngBytes),
}
payload, err := json.Marshal(body)
if err != nil {
return "", fmt.Errorf("marshal volcengine ocr request: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint.String(), bytes.NewReader(payload))
if err != nil {
return "", fmt.Errorf("build volcengine ocr request: %w", err)
}
req.Header.Set("Accept", "application/json")
req.Header.Set("Content-Type", "application/json")
c.signVolcengine(req, payload)
return c.doOCRRequest(req, "volcengine ocr")
}
func (c *Client) doOCRRequest(req *http.Request, label string) (string, error) {
resp, err := c.httpClient.Do(req)
if err != nil {
return "", fmt.Errorf("%s request: %w", label, err)
}
defer resp.Body.Close()
data, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return "", fmt.Errorf("%s status %d: %s", label, resp.StatusCode, compactBody(data))
}
text, err := parseOCRText(data)
if err != nil {
return "", fmt.Errorf("%s response: %w", label, err)
}
return text, nil
}
func (c *Client) signAliyun(req *http.Request, payload []byte) {
now := c.now().UTC()
payloadHash := sha256Hex(payload)
nonce := c.nonce()
req.Header.Set("x-acs-action", c.cfg.Action)
req.Header.Set("x-acs-version", c.cfg.Version)
req.Header.Set("x-acs-date", now.Format("2006-01-02T15:04:05Z"))
req.Header.Set("x-acs-signature-nonce", nonce)
req.Header.Set("x-acs-content-sha256", payloadHash)
signedHeaders := []string{
"content-type",
"host",
"x-acs-action",
"x-acs-content-sha256",
"x-acs-date",
"x-acs-signature-nonce",
"x-acs-version",
}
canonicalHeaders := canonicalHeaders(req, signedHeaders)
canonicalRequest := strings.Join([]string{
req.Method,
canonicalURI(req.URL),
canonicalQuery(req.URL),
canonicalHeaders,
strings.Join(signedHeaders, ";"),
payloadHash,
}, "\n")
stringToSign := "ACS3-HMAC-SHA256\n" + sha256Hex([]byte(canonicalRequest))
signature := hmacSHA256Hex([]byte(c.cfg.AccessKeySecret), []byte(stringToSign))
req.Header.Set(
"Authorization",
fmt.Sprintf("ACS3-HMAC-SHA256 Credential=%s,SignedHeaders=%s,Signature=%s",
c.cfg.AccessKeyID,
strings.Join(signedHeaders, ";"),
signature,
),
)
}
func (c *Client) signVolcengine(req *http.Request, payload []byte) {
now := c.now().UTC()
xDate := now.Format("20060102T150405Z")
shortDate := now.Format("20060102")
payloadHash := sha256Hex(payload)
req.Header.Set("X-Date", xDate)
req.Header.Set("X-Content-Sha256", payloadHash)
signedHeaders := []string{
"content-type",
"host",
"x-content-sha256",
"x-date",
}
canonicalHeaders := canonicalHeaders(req, signedHeaders)
canonicalRequest := strings.Join([]string{
req.Method,
canonicalURI(req.URL),
canonicalQuery(req.URL),
canonicalHeaders,
strings.Join(signedHeaders, ";"),
payloadHash,
}, "\n")
scope := strings.Join([]string{shortDate, c.cfg.Region, c.cfg.Service, "request"}, "/")
stringToSign := strings.Join([]string{
"HMAC-SHA256",
xDate,
scope,
sha256Hex([]byte(canonicalRequest)),
}, "\n")
signingKey := volcengineSigningKey(c.cfg.AccessKeySecret, shortDate, c.cfg.Region, c.cfg.Service)
signature := hmacSHA256Hex(signingKey, []byte(stringToSign))
req.Header.Set(
"Authorization",
fmt.Sprintf("HMAC-SHA256 Credential=%s/%s, SignedHeaders=%s, Signature=%s",
c.cfg.AccessKeyID,
scope,
strings.Join(signedHeaders, ";"),
signature,
),
)
}
func parseEndpoint(raw string) (*url.URL, error) {
endpoint := strings.TrimSpace(raw)
if endpoint == "" {
return nil, fmt.Errorf("ocr endpoint is required")
}
if !strings.Contains(endpoint, "://") {
endpoint = "https://" + endpoint
}
parsed, err := url.Parse(endpoint)
if err != nil {
return nil, fmt.Errorf("parse ocr endpoint: %w", err)
}
if parsed.Scheme == "" || parsed.Host == "" {
return nil, fmt.Errorf("invalid ocr endpoint %q", raw)
}
if parsed.Path == "" {
parsed.Path = "/"
}
return parsed, nil
}
func parseOCRText(data []byte) (string, error) {
var root any
if err := json.Unmarshal(data, &root); err != nil {
return "", fmt.Errorf("decode json: %w", err)
}
if msg := responseError(root); msg != "" {
return "", errors.New(msg)
}
parts := collectOCRParts(root, "")
if len(parts) == 0 {
return "", fmt.Errorf("no text found")
}
return strings.Join(dedupeNonEmpty(parts), "\n"), nil
}
func responseError(v any) string {
obj, ok := v.(map[string]any)
if !ok {
return ""
}
if meta, ok := obj["ResponseMetadata"].(map[string]any); ok {
if errObj, ok := meta["Error"].(map[string]any); ok {
if msg := stringValue(errObj["Message"]); msg != "" {
return msg
}
if code := stringValue(errObj["Code"]); code != "" {
return code
}
}
}
if errObj, ok := obj["Error"].(map[string]any); ok {
if msg := stringValue(errObj["Message"]); msg != "" {
return msg
}
if msg := stringValue(errObj["message"]); msg != "" {
return msg
}
}
if code, ok := numericCode(obj["code"]); ok && code != 0 && code != 10000 {
if msg := stringValue(obj["message"]); msg != "" {
return msg
}
if msg := stringValue(obj["msg"]); msg != "" {
return msg
}
return fmt.Sprintf("provider code %.0f", code)
}
if code := stringValue(obj["Code"]); code != "" && !isSuccessCode(code) {
if msg := stringValue(obj["Message"]); msg != "" {
return msg
}
return code
}
return ""
}
func collectOCRParts(v any, parentKey string) []string {
switch value := v.(type) {
case string:
text := strings.TrimSpace(value)
if text == "" {
return nil
}
if looksLikeJSON(text) {
var nested any
if err := json.Unmarshal([]byte(text), &nested); err == nil {
return collectOCRParts(nested, parentKey)
}
}
if isTextKey(parentKey) || isContainerTextKey(parentKey) {
return []string{text}
}
return nil
case []any:
parts := make([]string, 0, len(value))
for _, item := range value {
parts = append(parts, collectOCRParts(item, parentKey)...)
}
return parts
case map[string]any:
parts := make([]string, 0)
for _, key := range priorityOCRKeys() {
if child, ok := value[key]; ok {
parts = append(parts, collectOCRParts(child, key)...)
}
}
keys := make([]string, 0, len(value))
for key := range value {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
if isPriorityOCRKey(key) || isMetadataKey(key) {
continue
}
parts = append(parts, collectOCRParts(value[key], key)...)
}
return parts
default:
return nil
}
}
func priorityOCRKeys() []string {
return []string{
"Data",
"data",
"Result",
"result",
"content",
"Content",
"text",
"Text",
"DetectedText",
"detected_text",
"line_text",
"LineText",
"line_texts",
"LineTexts",
"word",
"Word",
"words",
"Words",
"words_result",
"WordsResult",
"prism_wordsInfo",
"prism_words_info",
"ocr_infos",
"OCRInfos",
"items",
"Items",
"blocks",
"Blocks",
"regions",
"Regions",
}
}
func isPriorityOCRKey(key string) bool {
for _, item := range priorityOCRKeys() {
if item == key {
return true
}
}
return false
}
func isTextKey(key string) bool {
switch normalizeKey(key) {
case "content", "text", "detectedtext", "linetext", "word", "words", "value":
return true
default:
return false
}
}
func isContainerTextKey(key string) bool {
switch normalizeKey(key) {
case "data", "result", "linetexts", "texts":
return true
default:
return false
}
}
func isMetadataKey(key string) bool {
switch normalizeKey(key) {
case "requestid", "request", "code", "status", "statuscode", "success",
"error", "errors", "message", "msg", "cost", "angle", "probability",
"confidence", "height", "width", "left", "top", "right", "bottom",
"x", "y", "responsemetadata":
return true
default:
return false
}
}
func normalizeKey(key string) string {
key = strings.ToLower(key)
key = strings.ReplaceAll(key, "_", "")
key = strings.ReplaceAll(key, "-", "")
return key
}
func dedupeNonEmpty(parts []string) []string {
seen := map[string]struct{}{}
out := make([]string, 0, len(parts))
for _, part := range parts {
for _, line := range strings.Split(part, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
if _, ok := seen[line]; ok {
continue
}
seen[line] = struct{}{}
out = append(out, line)
}
}
return out
}
func looksLikeJSON(text string) bool {
return strings.HasPrefix(text, "{") || strings.HasPrefix(text, "[")
}
func stringValue(v any) string {
if s, ok := v.(string); ok {
return strings.TrimSpace(s)
}
return ""
}
func numericCode(v any) (float64, bool) {
switch n := v.(type) {
case float64:
return n, true
case int:
return float64(n), true
case json.Number:
f, err := n.Float64()
return f, err == nil
default:
return 0, false
}
}
func isSuccessCode(code string) bool {
switch strings.ToLower(strings.TrimSpace(code)) {
case "", "ok", "success", "200", "10000":
return true
default:
return false
}
}
func canonicalURI(u *url.URL) string {
if u == nil || u.EscapedPath() == "" {
return "/"
}
return u.EscapedPath()
}
func canonicalQuery(u *url.URL) string {
if u == nil || u.RawQuery == "" {
return ""
}
values, _ := url.ParseQuery(u.RawQuery)
return values.Encode()
}
func canonicalHeaders(req *http.Request, signedHeaders []string) string {
lines := make([]string, 0, len(signedHeaders))
for _, key := range signedHeaders {
var value string
if key == "host" {
value = req.URL.Host
} else {
value = req.Header.Get(key)
}
lines = append(lines, key+":"+normalizeHeaderValue(value)+"\n")
}
return strings.Join(lines, "")
}
func normalizeHeaderValue(value string) string {
return strings.Join(strings.Fields(strings.TrimSpace(value)), " ")
}
func volcengineSigningKey(secret, date, region, service string) []byte {
kDate := hmacSHA256([]byte(secret), []byte(date))
kRegion := hmacSHA256(kDate, []byte(region))
kService := hmacSHA256(kRegion, []byte(service))
return hmacSHA256(kService, []byte("request"))
}
func hmacSHA256(key, data []byte) []byte {
mac := hmac.New(sha256.New, key)
mac.Write(data)
return mac.Sum(nil)
}
func hmacSHA256Hex(key, data []byte) string {
return hex.EncodeToString(hmacSHA256(key, data))
}
func sha256Hex(data []byte) string {
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}
func randomNonce() string {
buf := make([]byte, 16)
if _, err := rand.Read(buf); err != nil {
return fmt.Sprintf("%d", time.Now().UnixNano())
}
return hex.EncodeToString(buf)
}
func compactBody(data []byte) string {
text := strings.TrimSpace(string(data))
if text == "" {
return "<empty body>"
}
if len(text) > 800 {
return text[:800] + "..."
}
return text
}
+150
View File
@@ -0,0 +1,150 @@
package ocr
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"strings"
"testing"
"time"
"github.com/mmmy/snapgo/internal/domain"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func TestAliyunRecognizeTextSignsRawPNGRequest(t *testing.T) {
var requestBody []byte
transport := roundTripFunc(func(r *http.Request) (*http.Response, error) {
if r.URL.Host != "ocr-api.cn-hangzhou.aliyuncs.com" {
t.Fatalf("unexpected host %s", r.URL.Host)
}
if got := r.Header.Get("x-acs-action"); got != "RecognizeGeneral" {
t.Fatalf("unexpected action %q", got)
}
if got := r.Header.Get("x-acs-version"); got != "2021-07-07" {
t.Fatalf("unexpected version %q", got)
}
if got := r.Header.Get("x-acs-date"); got != "2026-07-08T01:02:03Z" {
t.Fatalf("unexpected date %q", got)
}
if got := r.Header.Get("Authorization"); !strings.Contains(got, "ACS3-HMAC-SHA256 Credential=ak") {
t.Fatalf("unexpected auth header %q", got)
}
var err error
requestBody, err = io.ReadAll(r.Body)
if err != nil {
t.Fatalf("read body: %v", err)
}
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(bytes.NewBufferString(
`{"Data":"{\"content\":\"第一行\\n第二行\"}"}`,
)),
}, nil
})
client, err := NewClient(domain.OCRProviderAliyun, domain.OCRProviderConfig{
Endpoint: "https://ocr-api.cn-hangzhou.aliyuncs.com",
AccessKeyID: "ak",
AccessKeySecret: "sk",
Action: "RecognizeGeneral",
Version: "2021-07-07",
})
if err != nil {
t.Fatalf("new client: %v", err)
}
client.httpClient.Transport = transport
client.now = func() time.Time { return time.Date(2026, 7, 8, 1, 2, 3, 0, time.UTC) }
client.nonce = func() string { return "nonce" }
text, err := client.RecognizeText(context.Background(), []byte("png"))
if err != nil {
t.Fatalf("recognize text: %v", err)
}
if string(requestBody) != "png" {
t.Fatalf("expected raw png body, got %q", string(requestBody))
}
if text != "第一行\n第二行" {
t.Fatalf("unexpected OCR text %q", text)
}
}
func TestVolcengineRecognizeTextSignsBase64JSONRequest(t *testing.T) {
var requestBody map[string]string
transport := roundTripFunc(func(r *http.Request) (*http.Response, error) {
if got := r.URL.Query().Get("Action"); got != "OCRNormal" {
t.Fatalf("unexpected action %q", got)
}
if got := r.URL.Query().Get("Version"); got != "2020-08-26" {
t.Fatalf("unexpected version %q", got)
}
if got := r.Header.Get("X-Date"); got != "20260708T010203Z" {
t.Fatalf("unexpected x-date %q", got)
}
if got := r.Header.Get("Authorization"); !strings.Contains(got, "HMAC-SHA256 Credential=ak/20260708/cn-north-1/cv/request") {
t.Fatalf("unexpected auth header %q", got)
}
if err := json.NewDecoder(r.Body).Decode(&requestBody); err != nil {
t.Fatalf("decode request: %v", err)
}
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(bytes.NewBufferString(
`{"ResponseMetadata":{"RequestId":"r"},"Result":{"LineTexts":["你好","世界"]}}`,
)),
}, nil
})
client, err := NewClient(domain.OCRProviderVolcengine, domain.OCRProviderConfig{
Endpoint: "https://visual.volcengineapi.com",
AccessKeyID: "ak",
AccessKeySecret: "sk",
Region: "cn-north-1",
Service: "cv",
Action: "OCRNormal",
Version: "2020-08-26",
})
if err != nil {
t.Fatalf("new client: %v", err)
}
client.httpClient.Transport = transport
client.now = func() time.Time { return time.Date(2026, 7, 8, 1, 2, 3, 0, time.UTC) }
text, err := client.RecognizeText(context.Background(), []byte("png"))
if err != nil {
t.Fatalf("recognize text: %v", err)
}
if requestBody["image_base64"] != base64.StdEncoding.EncodeToString([]byte("png")) {
t.Fatalf("expected base64 image body, got %#v", requestBody)
}
if text != "你好\n世界" {
t.Fatalf("unexpected OCR text %q", text)
}
}
func TestParseOCRTextSupportsWordsResult(t *testing.T) {
text, err := parseOCRText([]byte(`{"data":{"words_result":[{"words":"foo"},{"words":"bar"}]}}`))
if err != nil {
t.Fatalf("parse ocr text: %v", err)
}
if text != "foo\nbar" {
t.Fatalf("unexpected OCR text %q", text)
}
}
func TestParseOCRTextReturnsProviderError(t *testing.T) {
_, err := parseOCRText([]byte(`{"ResponseMetadata":{"Error":{"Code":"BadRequest","Message":"bad image"}}}`))
if err == nil || !strings.Contains(err.Error(), "bad image") {
t.Fatalf("expected provider error, got %v", err)
}
}
+14
View File
@@ -101,6 +101,20 @@ func (p *S3Provider) BuildPublicURL(key string) string {
return endpoint + "/" + p.cfg.Bucket + "/" + key return endpoint + "/" + p.cfg.Bucket + "/" + key
} }
// Delete removes a previously uploaded object by key.
//
// The summary pipeline uses this to clean up the temporary screenshot it had
// to upload only so a remote multimodal model could fetch it via public URL.
func (p *S3Provider) Delete(ctx context.Context, key string) error {
if _, err := p.client.DeleteObject(ctx, &s3.DeleteObjectInput{
Bucket: awsv2.String(p.cfg.Bucket),
Key: awsv2.String(key),
}); err != nil {
return fmt.Errorf("s3 delete object: %w", err)
}
return nil
}
// TestConnection performs a small put + delete to verify credentials and // TestConnection performs a small put + delete to verify credentials and
// bucket reachability. Used by the "Test connection" button in settings. // bucket reachability. Used by the "Test connection" button in settings.
func (p *S3Provider) TestConnection(ctx context.Context) error { func (p *S3Provider) TestConnection(ctx context.Context) error {
+3 -3
View File
@@ -33,10 +33,10 @@ func Start(cbs Callbacks) (start, stop func()) {
systray.SetTemplateIcon(templateIconBytes, regularIconBytes) systray.SetTemplateIcon(templateIconBytes, regularIconBytes)
systray.SetTooltip("SnapGo") systray.SetTooltip("SnapGo")
mCapture := systray.AddMenuItem("Capture screenshot", "Take a region screenshot") mCapture := systray.AddMenuItem("截图", "Take a region screenshot")
mSettings := systray.AddMenuItem("Settings…", "Open settings window") mSettings := systray.AddMenuItem("设置", "Open settings window")
systray.AddSeparator() systray.AddSeparator()
mQuit := systray.AddMenuItem("Quit SnapGo", "Quit the application") mQuit := systray.AddMenuItem("退出", "Quit the application")
// Pump menu clicks on a dedicated goroutine. Channel sends from // Pump menu clicks on a dedicated goroutine. Channel sends from
// systray are non-blocking, so a slow user callback does not back // systray are non-blocking, so a slow user callback does not back
+2 -2
View File
@@ -36,8 +36,8 @@ func main() {
err := wails.Run(&options.App{ err := wails.Run(&options.App{
Title: "SnapGo", Title: "SnapGo",
Width: 1080, Width: 1000,
Height: 720, Height: 820,
MinWidth: 720, MinWidth: 720,
MinHeight: 520, MinHeight: 520,
AssetServer: &assetserver.Options{ AssetServer: &assetserver.Options{
+24
View File
@@ -74,6 +74,30 @@ func nativeOverlaySaveRemote(x, y, w, h C.int, annotationsJSON *C.char) {
}() }()
} }
//export nativeOverlaySummarize
func nativeOverlaySummarize(x, y, w, h C.int, annotationsJSON *C.char) {
app := consumeNativeOverlayApp()
if app == nil {
return
}
result := nativeCaptureResult(x, y, w, h, annotationsJSON)
go func() {
_ = app.SummarizeNativeRegion(result)
}()
}
//export nativeOverlayOCR
func nativeOverlayOCR(x, y, w, h C.int, annotationsJSON *C.char) {
app := consumeNativeOverlayApp()
if app == nil {
return
}
result := nativeCaptureResult(x, y, w, h, annotationsJSON)
go func() {
_ = app.ExtractTextNativeRegion(result)
}()
}
//export nativeOverlayCancel //export nativeOverlayCancel
func nativeOverlayCancel() { func nativeOverlayCancel() {
app := consumeNativeOverlayApp() app := consumeNativeOverlayApp()
+198 -22
View File
@@ -15,6 +15,8 @@ extern void nativeOverlayConfirm(int x, int y, int w, int h, const char *annotat
extern void nativeOverlayCopy(int x, int y, int w, int h, const char *annotationsJSON); extern void nativeOverlayCopy(int x, int y, int w, int h, const char *annotationsJSON);
extern void nativeOverlaySave(int x, int y, int w, int h, const char *annotationsJSON, const char *dir); extern void nativeOverlaySave(int x, int y, int w, int h, const char *annotationsJSON, const char *dir);
extern void nativeOverlaySaveRemote(int x, int y, int w, int h, const char *annotationsJSON); extern void nativeOverlaySaveRemote(int x, int y, int w, int h, const char *annotationsJSON);
extern void nativeOverlaySummarize(int x, int y, int w, int h, const char *annotationsJSON);
extern void nativeOverlayOCR(int x, int y, int w, int h, const char *annotationsJSON);
extern void nativeOverlayCancel(void); extern void nativeOverlayCancel(void);
static NSWindow *nativeOverlayWindow = nil; static NSWindow *nativeOverlayWindow = nil;
@@ -101,26 +103,36 @@ static id nativeOverlayKeyMonitor = nil;
// as returning a +1 retained object. // as returning a +1 retained object.
@property(strong) NSButton *clipboardButton; @property(strong) NSButton *clipboardButton;
@property(strong) NSButton *saveButton; @property(strong) NSButton *saveButton;
// `saveRemoteButton` triggers a placeholder action ("not yet supported")
// per product spec; the icon comes from save-remote.svg.
@property(strong) NSButton *saveRemoteButton; @property(strong) NSButton *saveRemoteButton;
@property(strong) NSButton *ocrButton;
@property(strong) NSButton *summaryButton;
@property(strong) NSButton *uploadButton; @property(strong) NSButton *uploadButton;
@property(strong) NSButton *penButton; @property(strong) NSButton *penButton;
@property(strong) NSButton *rectButton; @property(strong) NSButton *rectButton;
@property(strong) NSButton *ellipseButton; @property(strong) NSButton *ellipseButton;
@property(strong) NSButton *textButton;
@property(strong) NSButton *colorButton; @property(strong) NSButton *colorButton;
@property(strong) NSButton *undoButton; @property(strong) NSButton *undoButton;
@property(strong) NSView *paletteView; @property(strong) NSView *paletteView;
@property(strong) NSView *actionToolbarBg; @property(strong) NSView *actionToolbarBg;
@property(strong) NSTextField *textEditor;
@property NSPoint textEditorLocalPoint;
@property(strong) NSTextField *sizeLabel; @property(strong) NSTextField *sizeLabel;
@property(strong) NSTextField *hintLabel; @property(strong) NSTextField *hintLabel;
- (void)syncControls; - (void)syncControls;
- (void)styleControls; - (void)styleControls;
- (void)selectText;
- (void)confirmSelection; - (void)confirmSelection;
- (void)copySelection; - (void)copySelection;
- (void)saveSelection; - (void)saveSelection;
- (void)saveRemoteSelection; - (void)saveRemoteSelection;
- (void)ocrSelection;
- (void)summarizeSelection;
- (void)cancelSelection; - (void)cancelSelection;
- (void)beginTextAnnotationAtPoint:(NSPoint)localPoint;
- (BOOL)isEditingText;
- (void)commitTextEditor;
- (void)cancelTextEditor;
@end @end
@implementation SnipNativeOverlayView @implementation SnipNativeOverlayView
@@ -137,13 +149,15 @@ static id nativeOverlayKeyMonitor = nil;
_activeColor = [NSColor colorWithCalibratedRed:239.0/255.0 green:68.0/255.0 blue:68.0/255.0 alpha:1.0]; _activeColor = [NSColor colorWithCalibratedRed:239.0/255.0 green:68.0/255.0 blue:68.0/255.0 alpha:1.0];
_annotations = [NSMutableArray array]; _annotations = [NSMutableArray array];
// Use SnipHoverButton for the 5 action buttons so we get hover-color // Use SnipHoverButton for the action buttons so we get hover-color
// animation. Annotation buttons stay as plain NSButton because they // animation. Annotation buttons stay as plain NSButton because they
// already have explicit on/off "active" styling. // already have explicit on/off "active" styling.
_cancelButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(cancelSelection)]; _cancelButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(cancelSelection)];
_clipboardButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(copySelection)]; _clipboardButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(copySelection)];
_saveButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(saveSelection)]; _saveButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(saveSelection)];
_saveRemoteButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(saveRemoteSelection)]; _saveRemoteButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(saveRemoteSelection)];
_ocrButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(ocrSelection)];
_summaryButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(summarizeSelection)];
_uploadButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(confirmSelection)]; _uploadButton = [SnipHoverButton buttonWithTitle:@"" target:self action:@selector(confirmSelection)];
// Tooltip strings shown on hover for each action button. // Tooltip strings shown on hover for each action button.
// Localized in Chinese to match the rest of the action UI surface. // Localized in Chinese to match the rest of the action UI surface.
@@ -151,10 +165,13 @@ static id nativeOverlayKeyMonitor = nil;
[_clipboardButton setToolTip:@"复制图片"]; [_clipboardButton setToolTip:@"复制图片"];
[_saveButton setToolTip:@"保存本地"]; [_saveButton setToolTip:@"保存本地"];
[_saveRemoteButton setToolTip:@"保存远端"]; [_saveRemoteButton setToolTip:@"保存远端"];
[_ocrButton setToolTip:@"提取文字"];
[_summaryButton setToolTip:@"复制总结"];
[_uploadButton setToolTip:@"上传云端"]; [_uploadButton setToolTip:@"上传云端"];
_penButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectPen)]; _penButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectPen)];
_rectButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectRect)]; _rectButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectRect)];
_ellipseButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectEllipse)]; _ellipseButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectEllipse)];
_textButton = [NSButton buttonWithTitle:@"" target:self action:@selector(selectText)];
_colorButton = [NSButton buttonWithTitle:@"" target:self action:@selector(togglePalette)]; _colorButton = [NSButton buttonWithTitle:@"" target:self action:@selector(togglePalette)];
_undoButton = [NSButton buttonWithTitle:@"" target:self action:@selector(undoAnnotation)]; _undoButton = [NSButton buttonWithTitle:@"" target:self action:@selector(undoAnnotation)];
_paletteView = [[NSView alloc] initWithFrame:NSZeroRect]; _paletteView = [[NSView alloc] initWithFrame:NSZeroRect];
@@ -171,10 +188,10 @@ static id nativeOverlayKeyMonitor = nil;
// Add the action toolbar background BEFORE the buttons so it sits // Add the action toolbar background BEFORE the buttons so it sits
// behind them in the view hierarchy (AppKit z-order = subview order). // behind them in the view hierarchy (AppKit z-order = subview order).
[self addSubview:_actionToolbarBg]; [self addSubview:_actionToolbarBg];
for (NSView *view in @[_cancelButton, _clipboardButton, _saveButton, _saveRemoteButton, _uploadButton, _penButton, _rectButton, _ellipseButton, _colorButton, _undoButton, _paletteView, _sizeLabel, _hintLabel]) { for (NSView *view in @[_cancelButton, _clipboardButton, _saveButton, _saveRemoteButton, _ocrButton, _summaryButton, _uploadButton, _penButton, _rectButton, _ellipseButton, _textButton, _colorButton, _undoButton, _paletteView, _sizeLabel, _hintLabel]) {
[self addSubview:view]; [self addSubview:view];
} }
for (NSView *view in @[_cancelButton, _clipboardButton, _saveButton, _saveRemoteButton, _uploadButton, _penButton, _rectButton, _ellipseButton, _colorButton, _undoButton, _paletteView, _actionToolbarBg]) { for (NSView *view in @[_cancelButton, _clipboardButton, _saveButton, _saveRemoteButton, _ocrButton, _summaryButton, _uploadButton, _penButton, _rectButton, _ellipseButton, _textButton, _colorButton, _undoButton, _paletteView, _actionToolbarBg]) {
[view setHidden:YES]; [view setHidden:YES];
} }
[_sizeLabel setHidden:YES]; [_sizeLabel setHidden:YES];
@@ -237,8 +254,8 @@ static id nativeOverlayKeyMonitor = nil;
} }
- (void)styleControls { - (void)styleControls {
// Cancel / Copy / Save / Save-remote / Upload — square icon buttons // Cancel / Copy / Save / Save-remote / OCR / Summary / Upload — square icon buttons
// rendered from the user-supplied SVG assets. All five default to white // rendered from the user-supplied SVG assets. All actions default to white
// and animate to a per-action accent color on hover (cancel = red, // and animate to a per-action accent color on hover (cancel = red,
// others = blue). The base "white default" replaces the previous blue // others = blue). The base "white default" replaces the previous blue
// upload tint so the toolbar reads as a uniform icon row. // upload tint so the toolbar reads as a uniform icon row.
@@ -248,6 +265,8 @@ static id nativeOverlayKeyMonitor = nil;
// save-remote.svg — provided by the user; its content is the same arrow // save-remote.svg — provided by the user; its content is the same arrow
// icon as the previous upload, so we reuse it verbatim here. // icon as the previous upload, so we reuse it verbatim here.
NSImage *saveRemoteIcon = [self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M481.749333 353.792c16.682667-16.64 43.690667-16.64 60.373334 0l120.917333 120.832 3.541333 4.010667a42.666667 42.666667 0 0 1-63.872 56.32L554.666667 486.954667V896a42.666667 42.666667 0 0 1-85.333334 0v-409.173333l-48.170666 48.128a42.666667 42.666667 0 0 1-60.330667 0l-3.541333-4.010667a42.666667 42.666667 0 0 1 3.541333-56.32zM512 85.333333c122.538667 0 227.84 73.813333 273.92 179.370667A213.333333 213.333333 0 0 1 725.333333 682.666667a42.666667 42.666667 0 0 1-4.992-85.034667L725.333333 597.333333a128 128 0 0 0 36.394667-250.794666l-38.144-11.264-15.914667-36.437334a213.376 213.376 0 0 0-407.296 58.026667l-7.381333 58.368-57.173333 13.824A85.418667 85.418667 0 0 0 256 597.333333h42.666667a42.666667 42.666667 0 0 1 0 85.333334H256a170.666667 170.666667 0 0 1-40.277333-336.554667A298.709333 298.709333 0 0 1 512 85.333333z' fill='white'/></svg>"]; NSImage *saveRemoteIcon = [self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M481.749333 353.792c16.682667-16.64 43.690667-16.64 60.373334 0l120.917333 120.832 3.541333 4.010667a42.666667 42.666667 0 0 1-63.872 56.32L554.666667 486.954667V896a42.666667 42.666667 0 0 1-85.333334 0v-409.173333l-48.170666 48.128a42.666667 42.666667 0 0 1-60.330667 0l-3.541333-4.010667a42.666667 42.666667 0 0 1 3.541333-56.32zM512 85.333333c122.538667 0 227.84 73.813333 273.92 179.370667A213.333333 213.333333 0 0 1 725.333333 682.666667a42.666667 42.666667 0 0 1-4.992-85.034667L725.333333 597.333333a128 128 0 0 0 36.394667-250.794666l-38.144-11.264-15.914667-36.437334a213.376 213.376 0 0 0-407.296 58.026667l-7.381333 58.368-57.173333 13.824A85.418667 85.418667 0 0 0 256 597.333333h42.666667a42.666667 42.666667 0 0 1 0 85.333334H256a170.666667 170.666667 0 0 1-40.277333-336.554667A298.709333 298.709333 0 0 1 512 85.333333z' fill='white'/></svg>"];
NSImage *ocrIcon = [self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M192 160h192v64H256v128h-64V160zM640 160h192v192h-64V224H640v-64zM256 672v128h128v64H192V672h64zM832 672v192H640v-64h128V672h64zM469.333333 288h85.333334l170.666666 448h-78.933333l-39.253333-106.666667H416.853333L377.6 736H298.666667l170.666666-448zM439.466667 565.333333h144.896L512 368.981333 439.466667 565.333333z' fill='white'/></svg>"];
NSImage *summaryIcon = [self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M256 128h448l192 192v576H256V128z m64 64v640h512V352H672V192H320z m416 45.248V288h50.752L736 237.248zM384 448h384v64H384v-64z m0 128h384v64H384v-64z m0 128h256v64H384v-64zM192 256v704h512v64H128V256h64z' fill='white'/></svg>"];
// upload.svg — newly replaced "send/cloud" icon supplied by the user. // upload.svg — newly replaced "send/cloud" icon supplied by the user.
NSImage *uploadIcon = [self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M550.528 544.128L640 539.776V678.4l178.005333-178.005333L640 322.389333v129.024l-72.618667 10.922667c-155.52 23.424-251.733333 73.557333-312.874666 164.437333 84.906667-50.602667 178.261333-76.928 296.021333-82.645333zM554.666667 629.376c-27.392 1.322667-53.034667 3.968-77.141334 7.808l-8.192 1.365333c-136.704 23.637333-224.810667 87.168-310.954666 176.128-26.154667 27.008-51.882667 59.648-51.882667 59.648-14.848 18.176-24.021333 14.037333-20.352-9.173333 0 0 4.394667-31.402667 11.690667-65.792C144.938667 577.066667 250.581333 423.68 554.666667 377.941333V209.066667a38.4 38.4 0 0 1 65.536-27.136l291.328 291.285333a38.4 38.4 0 0 1 0 54.314667l-291.328 291.285333A38.4 38.4 0 0 1 554.666667 791.637333v-162.261333z' fill='white'/></svg>"]; NSImage *uploadIcon = [self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M550.528 544.128L640 539.776V678.4l178.005333-178.005333L640 322.389333v129.024l-72.618667 10.922667c-155.52 23.424-251.733333 73.557333-312.874666 164.437333 84.906667-50.602667 178.261333-76.928 296.021333-82.645333zM554.666667 629.376c-27.392 1.322667-53.034667 3.968-77.141334 7.808l-8.192 1.365333c-136.704 23.637333-224.810667 87.168-310.954666 176.128-26.154667 27.008-51.882667 59.648-51.882667 59.648-14.848 18.176-24.021333 14.037333-20.352-9.173333 0 0 4.394667-31.402667 11.690667-65.792C144.938667 577.066667 250.581333 423.68 554.666667 377.941333V209.066667a38.4 38.4 0 0 1 65.536-27.136l291.328 291.285333a38.4 38.4 0 0 1 0 54.314667l-291.328 291.285333A38.4 38.4 0 0 1 554.666667 791.637333v-162.261333z' fill='white'/></svg>"];
@@ -255,9 +274,11 @@ static id nativeOverlayKeyMonitor = nil;
[self styleIconButton:_clipboardButton image:copyIcon]; [self styleIconButton:_clipboardButton image:copyIcon];
[self styleIconButton:_saveButton image:saveIcon]; [self styleIconButton:_saveButton image:saveIcon];
[self styleIconButton:_saveRemoteButton image:saveRemoteIcon]; [self styleIconButton:_saveRemoteButton image:saveRemoteIcon];
[self styleIconButton:_ocrButton image:ocrIcon];
[self styleIconButton:_summaryButton image:summaryIcon];
[self styleIconButton:_uploadButton image:uploadIcon]; [self styleIconButton:_uploadButton image:uploadIcon];
// Configure hover colors for the 5 action buttons. White is the shared // Configure hover colors for the action buttons. White is the shared
// base; cancel turns red on hover and the rest turn blue, providing a // base; cancel turns red on hover and the rest turn blue, providing a
// glance-able cue for destructive vs. constructive actions. // glance-able cue for destructive vs. constructive actions.
NSColor *baseWhite = [NSColor whiteColor]; NSColor *baseWhite = [NSColor whiteColor];
@@ -267,16 +288,21 @@ static id nativeOverlayKeyMonitor = nil;
SnipHoverButton *clipboardHB = (SnipHoverButton *)_clipboardButton; SnipHoverButton *clipboardHB = (SnipHoverButton *)_clipboardButton;
SnipHoverButton *saveHB = (SnipHoverButton *)_saveButton; SnipHoverButton *saveHB = (SnipHoverButton *)_saveButton;
SnipHoverButton *saveRemoteHB = (SnipHoverButton *)_saveRemoteButton; SnipHoverButton *saveRemoteHB = (SnipHoverButton *)_saveRemoteButton;
SnipHoverButton *ocrHB = (SnipHoverButton *)_ocrButton;
SnipHoverButton *summaryHB = (SnipHoverButton *)_summaryButton;
SnipHoverButton *uploadHB = (SnipHoverButton *)_uploadButton; SnipHoverButton *uploadHB = (SnipHoverButton *)_uploadButton;
cancelHB.baseColor = baseWhite; cancelHB.hoverColor = hoverRed; cancelHB.baseColor = baseWhite; cancelHB.hoverColor = hoverRed;
clipboardHB.baseColor = baseWhite; clipboardHB.hoverColor = hoverBlue; clipboardHB.baseColor = baseWhite; clipboardHB.hoverColor = hoverBlue;
saveHB.baseColor = baseWhite; saveHB.hoverColor = hoverBlue; saveHB.baseColor = baseWhite; saveHB.hoverColor = hoverBlue;
saveRemoteHB.baseColor = baseWhite; saveRemoteHB.hoverColor = hoverBlue; saveRemoteHB.baseColor = baseWhite; saveRemoteHB.hoverColor = hoverBlue;
ocrHB.baseColor = baseWhite; ocrHB.hoverColor = hoverBlue;
summaryHB.baseColor = baseWhite; summaryHB.hoverColor = hoverBlue;
uploadHB.baseColor = baseWhite; uploadHB.hoverColor = hoverBlue; uploadHB.baseColor = baseWhite; uploadHB.hoverColor = hoverBlue;
[self styleIconButton:_penButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M742.72 752.064c-29.696 28.576-42.976 48.96-42.976 77.12 0 98.4 143.776 142.464 229.728 65.536l-21.344-23.84c-66.976 59.936-176.384 26.4-176.384-41.696 0-16.832 9.28-31.104 33.184-54.08l13.44-12.736c61.024-58.016 75.328-102.72 29.312-179.84-43.84-73.44-96.8-88.288-162.784-50.56-47.232 27.04-68.64 47.456-208.32 190.4-70.624 72.288-153.92 77.088-217.344 26.784-59.712-47.328-79.872-127.36-42.176-188.64 24.512-39.84 62.656-72.896 136.64-124.48 5.952-4.192 27.04-18.816 31.104-21.664 12.16-8.448 21.536-15.104 30.4-21.536 107.552-78.272 140.8-139.136 86.048-219.904-76.992-113.472-202.304-90.016-367.744 61.472l21.6 23.616c153.088-140.224 256.896-159.616 319.68-67.104 41.632 61.408 16.896 106.688-78.4 176.032-8.64 6.304-17.92 12.832-29.888 21.184l-31.136 21.632c-77.504 54.08-117.984 89.184-145.536 133.984-46.784 76.032-22.176 173.664 49.536 230.496 76.224 60.416 177.952 54.56 260.096-29.504 136.16-139.36 158.08-160.224 201.312-184.96 50.656-28.96 84.416-19.52 119.424 39.168 36.896 61.824 27.52 91.392-23.808 140.16 0.096-0.064-10.976 10.368-13.632 12.96z' fill='white'/></svg>"]]; [self styleIconButton:_penButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M742.72 752.064c-29.696 28.576-42.976 48.96-42.976 77.12 0 98.4 143.776 142.464 229.728 65.536l-21.344-23.84c-66.976 59.936-176.384 26.4-176.384-41.696 0-16.832 9.28-31.104 33.184-54.08l13.44-12.736c61.024-58.016 75.328-102.72 29.312-179.84-43.84-73.44-96.8-88.288-162.784-50.56-47.232 27.04-68.64 47.456-208.32 190.4-70.624 72.288-153.92 77.088-217.344 26.784-59.712-47.328-79.872-127.36-42.176-188.64 24.512-39.84 62.656-72.896 136.64-124.48 5.952-4.192 27.04-18.816 31.104-21.664 12.16-8.448 21.536-15.104 30.4-21.536 107.552-78.272 140.8-139.136 86.048-219.904-76.992-113.472-202.304-90.016-367.744 61.472l21.6 23.616c153.088-140.224 256.896-159.616 319.68-67.104 41.632 61.408 16.896 106.688-78.4 176.032-8.64 6.304-17.92 12.832-29.888 21.184l-31.136 21.632c-77.504 54.08-117.984 89.184-145.536 133.984-46.784 76.032-22.176 173.664 49.536 230.496 76.224 60.416 177.952 54.56 260.096-29.504 136.16-139.36 158.08-160.224 201.312-184.96 50.656-28.96 84.416-19.52 119.424 39.168 36.896 61.824 27.52 91.392-23.808 140.16 0.096-0.064-10.976 10.368-13.632 12.96z' fill='white'/></svg>"]];
[self styleIconButton:_rectButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M96 96h832v832H96V96z m32 32v768h768V128H128z' fill='white'/></svg>"]]; [self styleIconButton:_rectButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M96 96h832v832H96V96z m32 32v768h768V128H128z' fill='white'/></svg>"]];
[self styleIconButton:_ellipseButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M512 928c229.76 0 416-186.24 416-416S741.76 96 512 96 96 282.24 96 512s186.24 416 416 416z m0-32C299.936 896 128 724.064 128 512S299.936 128 512 128s384 171.936 384 384-171.936 384-384 384z' fill='white'/></svg>"]]; [self styleIconButton:_ellipseButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M512 928c229.76 0 416-186.24 416-416S741.76 96 512 96 96 282.24 96 512s186.24 416 416 416z m0-32C299.936 896 128 724.064 128 512S299.936 128 512 128s384 171.936 384 384-171.936 384-384 384z' fill='white'/></svg>"]];
[self styleIconButton:_textButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><path d='M224 160h576v96H560v608h-96V256H224V160zM384 864h256v64H384v-64z' fill='white'/></svg>"]];
[self styleIconButton:_colorButton image:nil]; [self styleIconButton:_colorButton image:nil];
[self styleIconButton:_undoButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><g transform='translate(1024 0) scale(-1 1)'><path d='M838.976 288l-169.344-162.56a16 16 0 1 1 22.144-23.04l213.632 204.992-213.312 215.84a16 16 0 0 1-22.784-22.464L847.936 320H454.56c-164.192 0-297.28 128.96-297.28 288s133.088 288 297.28 288H704v32h-242.784C266.88 928 128 784.736 128 608S266.88 288 461.248 288h377.728z' fill='white'/></g></svg>"]]; [self styleIconButton:_undoButton image:[self iconFromSVG:@"<svg viewBox='0 0 1024 1024' xmlns='http://www.w3.org/2000/svg'><g transform='translate(1024 0) scale(-1 1)'><path d='M838.976 288l-169.344-162.56a16 16 0 1 1 22.144-23.04l213.632 204.992-213.312 215.84a16 16 0 0 1-22.784-22.464L847.936 320H454.56c-164.192 0-297.28 128.96-297.28 288s133.088 288 297.28 288H704v32h-242.784C266.88 928 128 784.736 128 608S266.88 288 461.248 288h377.728z' fill='white'/></g></svg>"]];
[_paletteView setWantsLayer:YES]; [_paletteView setWantsLayer:YES];
@@ -359,6 +385,19 @@ static id nativeOverlayKeyMonitor = nil;
if (points.count == 0) { if (points.count == 0) {
continue; continue;
} }
if ([tool isEqualToString:@"text"]) {
NSString *text = item[@"text"];
if (text.length == 0) {
continue;
}
NSPoint p = [points[0] pointValue];
NSDictionary *attrs = @{
NSFontAttributeName: [NSFont systemFontOfSize:20 weight:NSFontWeightSemibold],
NSForegroundColorAttributeName: color ?: [NSColor whiteColor]
};
[text drawAtPoint:NSMakePoint(_selection.origin.x + p.x, _selection.origin.y + p.y) withAttributes:attrs];
continue;
}
[color setStroke]; [color setStroke];
NSBezierPath *path = [NSBezierPath bezierPath]; NSBezierPath *path = [NSBezierPath bezierPath];
[path setLineWidth:3]; [path setLineWidth:3];
@@ -468,9 +507,8 @@ static id nativeOverlayKeyMonitor = nil;
- (void)mouseDown:(NSEvent *)event { - (void)mouseDown:(NSEvent *)event {
NSPoint p = [self convertPoint:[event locationInWindow] fromView:nil]; NSPoint p = [self convertPoint:[event locationInWindow] fromView:nil];
if ([event clickCount] == 2 && _hasSelection && NSPointInRect(p, _selection)) { if (_textEditor != nil) {
[self copySelection]; [self commitTextEditor];
return;
} }
NSString *handle = [self resizeHandleAtPoint:p]; NSString *handle = [self resizeHandleAtPoint:p];
if (handle != nil) { if (handle != nil) {
@@ -480,7 +518,23 @@ static id nativeOverlayKeyMonitor = nil;
_annotating = NO; _annotating = NO;
_resizeHandle = handle; _resizeHandle = handle;
_resizeStart = _selection; _resizeStart = _selection;
} else if (_hasSelection && NSPointInRect(p, _selection) && _activeTool != nil) { [self syncControls];
return;
}
if (_hasSelection && NSPointInRect(p, _selection) && [_activeTool isEqualToString:@"text"]) {
_creating = NO;
_moving = NO;
_resizing = NO;
_annotating = NO;
[self beginTextAnnotationAtPoint:[self localPoint:p]];
[self syncControls];
return;
}
if ([event clickCount] == 2 && _hasSelection && NSPointInRect(p, _selection)) {
[self copySelection];
return;
}
if (_hasSelection && NSPointInRect(p, _selection) && _activeTool != nil) {
_annotating = YES; _annotating = YES;
_creating = NO; _creating = NO;
_moving = NO; _moving = NO;
@@ -572,6 +626,16 @@ static id nativeOverlayKeyMonitor = nil;
} }
- (void)keyDown:(NSEvent *)event { - (void)keyDown:(NSEvent *)event {
if ([self isEditingText]) {
if ([event keyCode] == 53 || [[event charactersIgnoringModifiers] isEqualToString:@"\033"]) {
[self cancelTextEditor];
return;
}
if ([event keyCode] == 36 || [[event charactersIgnoringModifiers] isEqualToString:@"\r"]) {
[self commitTextEditor];
return;
}
}
if ([event keyCode] == 53 || [[event charactersIgnoringModifiers] isEqualToString:@"\033"]) { if ([event keyCode] == 53 || [[event charactersIgnoringModifiers] isEqualToString:@"\033"]) {
[self cancelSelection]; [self cancelSelection];
} else if (([event keyCode] == 36 || [[event charactersIgnoringModifiers] isEqualToString:@"\r"]) && _hasSelection) { } else if (([event keyCode] == 36 || [[event charactersIgnoringModifiers] isEqualToString:@"\r"]) && _hasSelection) {
@@ -591,11 +655,14 @@ static id nativeOverlayKeyMonitor = nil;
[_clipboardButton setHidden:!visible]; [_clipboardButton setHidden:!visible];
[_saveButton setHidden:!visible]; [_saveButton setHidden:!visible];
[_saveRemoteButton setHidden:!visible]; [_saveRemoteButton setHidden:!visible];
[_ocrButton setHidden:!visible];
[_summaryButton setHidden:!visible];
[_uploadButton setHidden:!visible]; [_uploadButton setHidden:!visible];
[_actionToolbarBg setHidden:!visible]; [_actionToolbarBg setHidden:!visible];
[_penButton setHidden:!visible]; [_penButton setHidden:!visible];
[_rectButton setHidden:!visible]; [_rectButton setHidden:!visible];
[_ellipseButton setHidden:!visible]; [_ellipseButton setHidden:!visible];
[_textButton setHidden:!visible];
[_colorButton setHidden:!visible]; [_colorButton setHidden:!visible];
[_undoButton setHidden:!visible]; [_undoButton setHidden:!visible];
[_sizeLabel setHidden:!visible]; [_sizeLabel setHidden:!visible];
@@ -608,12 +675,12 @@ static id nativeOverlayKeyMonitor = nil;
[_sizeLabel setStringValue:[NSString stringWithFormat:@"%.0f × %.0f", _selection.size.width, _selection.size.height]]; [_sizeLabel setStringValue:[NSString stringWithFormat:@"%.0f × %.0f", _selection.size.width, _selection.size.height]];
[_sizeLabel setFrame:NSMakeRect(_selection.origin.x, MAX(0, _selection.origin.y - 26), 110, 22)]; [_sizeLabel setFrame:NSMakeRect(_selection.origin.x, MAX(0, _selection.origin.y - 26), 110, 22)];
// Action toolbar layout — 5 icon buttons, each 28x28, separated by 8px, // Action toolbar layout — 7 icon buttons, each 28x28, separated by 8px,
// with 4px outer padding. Total = 5*28 + 4*8 + 2*4 = 180px wide. // with 4px outer padding. Total = 7*28 + 6*8 + 2*4 = 252px wide.
CGFloat actionBtnSize = 28; CGFloat actionBtnSize = 28;
CGFloat actionGap = 8; CGFloat actionGap = 8;
CGFloat actionPad = 4; CGFloat actionPad = 4;
NSInteger actionCount = 5; NSInteger actionCount = 7;
CGFloat toolbarW = actionPad * 2 + actionBtnSize * actionCount + actionGap * (actionCount - 1); CGFloat toolbarW = actionPad * 2 + actionBtnSize * actionCount + actionGap * (actionCount - 1);
CGFloat toolbarH = 40; CGFloat toolbarH = 40;
CGFloat x = _selection.origin.x + _selection.size.width - toolbarW; CGFloat x = _selection.origin.x + _selection.size.width - toolbarW;
@@ -631,15 +698,21 @@ static id nativeOverlayKeyMonitor = nil;
[_clipboardButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 1, btnY, actionBtnSize, actionBtnSize)]; [_clipboardButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 1, btnY, actionBtnSize, actionBtnSize)];
[_saveButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 2, btnY, actionBtnSize, actionBtnSize)]; [_saveButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 2, btnY, actionBtnSize, actionBtnSize)];
[_saveRemoteButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 3, btnY, actionBtnSize, actionBtnSize)]; [_saveRemoteButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 3, btnY, actionBtnSize, actionBtnSize)];
[_uploadButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 4, btnY, actionBtnSize, actionBtnSize)]; [_ocrButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 4, btnY, actionBtnSize, actionBtnSize)];
[_summaryButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 5, btnY, actionBtnSize, actionBtnSize)];
[_uploadButton setFrame:NSMakeRect(baseX + (actionBtnSize + actionGap) * 6, btnY, actionBtnSize, actionBtnSize)];
CGFloat markW = 170; // Mark toolbar sits to the LEFT of the action toolbar (same row) with an
CGFloat markX = MAX(0, MIN(_selection.origin.x, self.bounds.size.width - markW)); // 8px gap between the two groups, so they never overlap on tiny selections.
CGFloat markW = 202;
CGFloat markGap = 8;
CGFloat markX = MAX(0, x - markGap - markW);
[_penButton setFrame:NSMakeRect(markX + 4, y + 4, 28, 28)]; [_penButton setFrame:NSMakeRect(markX + 4, y + 4, 28, 28)];
[_rectButton setFrame:NSMakeRect(markX + 36, y + 4, 28, 28)]; [_rectButton setFrame:NSMakeRect(markX + 36, y + 4, 28, 28)];
[_ellipseButton setFrame:NSMakeRect(markX + 68, y + 4, 28, 28)]; [_ellipseButton setFrame:NSMakeRect(markX + 68, y + 4, 28, 28)];
[_colorButton setFrame:NSMakeRect(markX + 104, y + 7, 22, 22)]; [_textButton setFrame:NSMakeRect(markX + 100, y + 4, 28, 28)];
[_undoButton setFrame:NSMakeRect(markX + 134, y + 4, 28, 28)]; [_colorButton setFrame:NSMakeRect(markX + 136, y + 7, 22, 22)];
[_undoButton setFrame:NSMakeRect(markX + 166, y + 4, 28, 28)];
[[_colorButton layer] setBackgroundColor:[_activeColor CGColor]]; [[_colorButton layer] setBackgroundColor:[_activeColor CGColor]];
[_undoButton setEnabled:_annotations.count > 0]; [_undoButton setEnabled:_annotations.count > 0];
[_paletteView setFrame:NSMakeRect(markX, MAX(0, y - 78), 150, 70)]; [_paletteView setFrame:NSMakeRect(markX, MAX(0, y - 78), 150, 70)];
@@ -647,7 +720,7 @@ static id nativeOverlayKeyMonitor = nil;
} }
- (void)updateToolButtonStates { - (void)updateToolButtonStates {
NSDictionary *buttons = @{@"pen": _penButton, @"rect": _rectButton, @"ellipse": _ellipseButton}; NSDictionary *buttons = @{@"pen": _penButton, @"rect": _rectButton, @"ellipse": _ellipseButton, @"text": _textButton};
for (NSString *tool in buttons) { for (NSString *tool in buttons) {
NSButton *button = buttons[tool]; NSButton *button = buttons[tool];
NSColor *bg = [tool isEqualToString:_activeTool] NSColor *bg = [tool isEqualToString:_activeTool]
@@ -664,7 +737,9 @@ static id nativeOverlayKeyMonitor = nil;
- (void)selectPen { [self toggleTool:@"pen"]; } - (void)selectPen { [self toggleTool:@"pen"]; }
- (void)selectRect { [self toggleTool:@"rect"]; } - (void)selectRect { [self toggleTool:@"rect"]; }
- (void)selectEllipse { [self toggleTool:@"ellipse"]; } - (void)selectEllipse { [self toggleTool:@"ellipse"]; }
- (void)selectText { [self toggleTool:@"text"]; }
- (void)undoAnnotation { - (void)undoAnnotation {
[self cancelTextEditor];
if (_annotations.count > 0) { if (_annotations.count > 0) {
[_annotations removeLastObject]; [_annotations removeLastObject];
[self syncControls]; [self syncControls];
@@ -672,6 +747,68 @@ static id nativeOverlayKeyMonitor = nil;
} }
} }
- (void)beginTextAnnotationAtPoint:(NSPoint)localPoint {
[self cancelTextEditor];
_textEditorLocalPoint = localPoint;
NSRect frame = NSMakeRect(_selection.origin.x + localPoint.x, _selection.origin.y + localPoint.y, 180, 30);
CGFloat maxX = self.bounds.size.width - frame.size.width - 8;
CGFloat maxY = self.bounds.size.height - frame.size.height - 8;
frame.origin.x = MAX(8, MIN(frame.origin.x, maxX));
frame.origin.y = MAX(8, MIN(frame.origin.y, maxY));
_textEditor = [[NSTextField alloc] initWithFrame:frame];
[_textEditor setBezeled:YES];
[_textEditor setBordered:YES];
[_textEditor setDrawsBackground:YES];
[_textEditor setBackgroundColor:[NSColor colorWithCalibratedWhite:0.08 alpha:0.78]];
[_textEditor setTextColor:_activeColor];
[_textEditor setFont:[NSFont systemFontOfSize:20 weight:NSFontWeightSemibold]];
[_textEditor setFocusRingType:NSFocusRingTypeNone];
[_textEditor setTarget:self];
[_textEditor setAction:@selector(commitTextEditorFromSender:)];
[self addSubview:_textEditor];
[[self window] makeFirstResponder:_textEditor];
}
- (BOOL)isEditingText {
return _textEditor != nil;
}
- (void)commitTextEditorFromSender:(id)sender {
[self commitTextEditor];
}
- (void)commitTextEditor {
if (_textEditor == nil) {
return;
}
NSString *text = [[_textEditor stringValue] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
if (text.length > 0) {
[_annotations addObject:[@{
@"tool": @"text",
@"color": [self hexForColor:_activeColor],
@"nsColor": _activeColor,
@"points": [NSMutableArray arrayWithObject:[NSValue valueWithPoint:_textEditorLocalPoint]],
@"text": text
} mutableCopy]];
}
[_textEditor removeFromSuperview];
_textEditor = nil;
[[self window] makeFirstResponder:self];
[self syncControls];
[self setNeedsDisplay:YES];
}
- (void)cancelTextEditor {
if (_textEditor == nil) {
return;
}
[_textEditor removeFromSuperview];
_textEditor = nil;
[[self window] makeFirstResponder:self];
[self setNeedsDisplay:YES];
}
- (void)togglePalette { - (void)togglePalette {
[_paletteView setHidden:![_paletteView isHidden]]; [_paletteView setHidden:![_paletteView isHidden]];
if (![_paletteView isHidden] && _paletteView.subviews.count == 0) { if (![_paletteView isHidden] && _paletteView.subviews.count == 0) {
@@ -710,6 +847,7 @@ static id nativeOverlayKeyMonitor = nil;
} }
- (NSString *)annotationsJSON { - (NSString *)annotationsJSON {
[self commitTextEditor];
NSMutableArray *payload = [NSMutableArray array]; NSMutableArray *payload = [NSMutableArray array];
for (NSDictionary *item in _annotations) { for (NSDictionary *item in _annotations) {
NSMutableArray *points = [NSMutableArray array]; NSMutableArray *points = [NSMutableArray array];
@@ -717,7 +855,12 @@ static id nativeOverlayKeyMonitor = nil;
NSPoint p = [value pointValue]; NSPoint p = [value pointValue];
[points addObject:@{@"x": @(p.x), @"y": @(p.y)}]; [points addObject:@{@"x": @(p.x), @"y": @(p.y)}];
} }
[payload addObject:@{@"tool": item[@"tool"], @"color": item[@"color"], @"points": points}]; NSMutableDictionary *entry = [@{@"tool": item[@"tool"], @"color": item[@"color"], @"points": points} mutableCopy];
NSString *text = item[@"text"];
if (text.length > 0) {
entry[@"text"] = text;
}
[payload addObject:entry];
} }
NSData *data = [NSJSONSerialization dataWithJSONObject:payload options:0 error:nil]; NSData *data = [NSJSONSerialization dataWithJSONObject:payload options:0 error:nil];
if (data == nil) { if (data == nil) {
@@ -780,6 +923,7 @@ static id nativeOverlayKeyMonitor = nil;
} }
- (void)cancelSelection { - (void)cancelSelection {
[self cancelTextEditor];
[self closeOverlayWindow]; [self closeOverlayWindow];
nativeOverlayCancel(); nativeOverlayCancel();
} }
@@ -796,6 +940,30 @@ static id nativeOverlayKeyMonitor = nil;
NSString *json = [self annotationsJSON]; NSString *json = [self annotationsJSON];
nativeOverlaySaveRemote((int)llround(r.origin.x), (int)llround(r.origin.y), (int)llround(r.size.width), (int)llround(r.size.height), [json UTF8String]); nativeOverlaySaveRemote((int)llround(r.origin.x), (int)llround(r.origin.y), (int)llround(r.size.width), (int)llround(r.size.height), [json UTF8String]);
} }
// `ocrSelection` sends the screenshot to the configured OCR provider and
// copies the extracted text.
- (void)ocrSelection {
if (!_hasSelection) {
return;
}
NSRect r = [self globalRectForSelection:_selection];
[self closeOverlayWindow];
NSString *json = [self annotationsJSON];
nativeOverlayOCR((int)llround(r.origin.x), (int)llround(r.origin.y), (int)llround(r.size.width), (int)llround(r.size.height), [json UTF8String]);
}
// `summarizeSelection` uploads the screenshot to S3, sends it to the
// configured multimodal model, and copies the generated summary.
- (void)summarizeSelection {
if (!_hasSelection) {
return;
}
NSRect r = [self globalRectForSelection:_selection];
[self closeOverlayWindow];
NSString *json = [self annotationsJSON];
nativeOverlaySummarize((int)llround(r.origin.x), (int)llround(r.origin.y), (int)llround(r.size.width), (int)llround(r.size.height), [json UTF8String]);
}
@end @end
// screenContainingCursor returns the NSScreen the cursor is currently on, // screenContainingCursor returns the NSScreen the cursor is currently on,
@@ -856,10 +1024,18 @@ static void snipShowNativeOverlay(int width, int height) {
nativeOverlayKeyMonitor = [NSEvent addLocalMonitorForEventsMatchingMask:NSEventMaskKeyDown handler:^NSEvent *(NSEvent *event) { nativeOverlayKeyMonitor = [NSEvent addLocalMonitorForEventsMatchingMask:NSEventMaskKeyDown handler:^NSEvent *(NSEvent *event) {
if ([event keyCode] == 53) { if ([event keyCode] == 53) {
if ([view isEditingText]) {
[view cancelTextEditor];
return nil;
}
[view cancelSelection]; [view cancelSelection];
return nil; return nil;
} }
if ([event keyCode] == 36 && [view hasSelection]) { if ([event keyCode] == 36 && [view hasSelection]) {
if ([view isEditingText]) {
[view commitTextEditor];
return nil;
}
[view confirmSelection]; [view confirmSelection];
return nil; return nil;
} }
+163
View File
@@ -0,0 +1,163 @@
//go:build darwin
package main
/*
#cgo CFLAGS: -x objective-c -fobjc-arc -fblocks
#cgo LDFLAGS: -framework AppKit -framework QuartzCore
#include <stdlib.h>
#include <dispatch/dispatch.h>
#import <AppKit/AppKit.h>
#import <QuartzCore/QuartzCore.h>
static NSPanel *operationPanel = nil;
static NSTextField *operationTitleLabel = nil;
static NSTextField *operationDetailLabel = nil;
static NSTextField *operationIconLabel = nil;
static NSProgressIndicator *operationSpinner = nil;
static NSString *statusString(const char *value) {
if (value == NULL) {
return @"";
}
NSString *text = [NSString stringWithUTF8String:value];
return text ?: @"";
}
static void ensureOperationPanel(void) {
if (operationPanel != nil) {
return;
}
NSRect frame = NSMakeRect(0, 0, 360, 118);
operationPanel = [[NSPanel alloc]
initWithContentRect:frame
styleMask:NSWindowStyleMaskBorderless
backing:NSBackingStoreBuffered
defer:NO];
[operationPanel setOpaque:NO];
[operationPanel setBackgroundColor:[NSColor clearColor]];
[operationPanel setLevel:NSFloatingWindowLevel];
[operationPanel setHidesOnDeactivate:NO];
[operationPanel setCollectionBehavior:
NSWindowCollectionBehaviorCanJoinAllSpaces |
NSWindowCollectionBehaviorFullScreenAuxiliary |
NSWindowCollectionBehaviorStationary];
NSView *root = [[NSView alloc] initWithFrame:frame];
[root setWantsLayer:YES];
[[root layer] setCornerRadius:14];
[[root layer] setBackgroundColor:[[NSColor colorWithCalibratedWhite:0.08 alpha:0.92] CGColor]];
[[root layer] setShadowColor:[[NSColor blackColor] CGColor]];
[[root layer] setShadowOpacity:0.28];
[[root layer] setShadowRadius:18];
[[root layer] setShadowOffset:CGSizeMake(0, -8)];
operationSpinner = [[NSProgressIndicator alloc] initWithFrame:NSMakeRect(24, 42, 34, 34)];
[operationSpinner setStyle:NSProgressIndicatorStyleSpinning];
[operationSpinner setIndeterminate:YES];
operationIconLabel = [NSTextField labelWithString:@"OK"];
[operationIconLabel setFrame:NSMakeRect(21, 43, 42, 30)];
[operationIconLabel setAlignment:NSTextAlignmentCenter];
[operationIconLabel setFont:[NSFont systemFontOfSize:17 weight:NSFontWeightSemibold]];
[operationIconLabel setTextColor:[NSColor colorWithCalibratedRed:74.0/255.0 green:222.0/255.0 blue:128.0/255.0 alpha:1.0]];
[operationIconLabel setHidden:YES];
operationTitleLabel = [NSTextField labelWithString:@""];
[operationTitleLabel setFrame:NSMakeRect(76, 61, 260, 24)];
[operationTitleLabel setFont:[NSFont systemFontOfSize:15 weight:NSFontWeightSemibold]];
[operationTitleLabel setTextColor:[NSColor whiteColor]];
operationDetailLabel = [NSTextField labelWithString:@""];
[operationDetailLabel setFrame:NSMakeRect(76, 34, 260, 22)];
[operationDetailLabel setFont:[NSFont systemFontOfSize:12 weight:NSFontWeightRegular]];
[operationDetailLabel setTextColor:[NSColor colorWithCalibratedWhite:0.82 alpha:1.0]];
[operationDetailLabel setLineBreakMode:NSLineBreakByTruncatingTail];
[root addSubview:operationSpinner];
[root addSubview:operationIconLabel];
[root addSubview:operationTitleLabel];
[root addSubview:operationDetailLabel];
[operationPanel setContentView:root];
}
static void snipShowOperationStatus(const char *titleC, const char *detailC, int state) {
NSString *title = statusString(titleC);
NSString *detail = statusString(detailC);
dispatch_async(dispatch_get_main_queue(), ^{
ensureOperationPanel();
[operationTitleLabel setStringValue:title];
[operationDetailLabel setStringValue:detail];
if (state == 0) {
[operationIconLabel setHidden:YES];
[operationSpinner setHidden:NO];
[operationSpinner startAnimation:nil];
} else {
[operationSpinner stopAnimation:nil];
[operationSpinner setHidden:YES];
[operationIconLabel setHidden:NO];
if (state == 2) {
[operationIconLabel setStringValue:@"!"];
[operationIconLabel setTextColor:[NSColor colorWithCalibratedRed:248.0/255.0 green:113.0/255.0 blue:113.0/255.0 alpha:1.0]];
} else {
[operationIconLabel setStringValue:@"OK"];
[operationIconLabel setTextColor:[NSColor colorWithCalibratedRed:74.0/255.0 green:222.0/255.0 blue:128.0/255.0 alpha:1.0]];
}
}
NSScreen *screen = [NSScreen mainScreen];
if (screen != nil) {
NSRect visible = [screen visibleFrame];
NSRect panelFrame = [operationPanel frame];
panelFrame.origin.x = NSMidX(visible) - panelFrame.size.width / 2;
panelFrame.origin.y = NSMaxY(visible) - panelFrame.size.height - 48;
[operationPanel setFrame:panelFrame display:YES];
}
[operationPanel orderFrontRegardless];
});
}
static void snipHideOperationStatus(void) {
dispatch_async(dispatch_get_main_queue(), ^{
if (operationPanel != nil) {
[operationSpinner stopAnimation:nil];
[operationPanel orderOut:nil];
}
});
}
*/
import "C"
import (
"sync/atomic"
"time"
"unsafe"
)
const (
operationStatusRunning = iota
operationStatusSuccess
operationStatusError
)
var operationStatusSeq atomic.Uint64
func showOperationStatus(title, detail string, state int) {
operationStatusSeq.Add(1)
ctitle := C.CString(title)
cdetail := C.CString(detail)
defer C.free(unsafe.Pointer(ctitle))
defer C.free(unsafe.Pointer(cdetail))
C.snipShowOperationStatus(ctitle, cdetail, C.int(state))
}
func hideOperationStatusAfter(delay time.Duration) {
seq := operationStatusSeq.Load()
time.AfterFunc(delay, func() {
if operationStatusSeq.Load() == seq {
C.snipHideOperationStatus()
}
})
}
+15
View File
@@ -0,0 +1,15 @@
//go:build !darwin
package main
import "time"
const (
operationStatusRunning = iota
operationStatusSuccess
operationStatusError
)
func showOperationStatus(_ string, _ string, _ int) {}
func hideOperationStatusAfter(_ time.Duration) {}
+3
View File
@@ -43,6 +43,9 @@ sleep 0.4
# --identity flag in v2. # --identity flag in v2.
ARCH="${ARCH:-arm64}" ARCH="${ARCH:-arm64}"
echo "[dev-build.sh] building darwin/${ARCH}..." echo "[dev-build.sh] building darwin/${ARCH}..."
# Allow the -Wl,-no_warn_duplicate_libraries flag declared in the darwin cgo
# files past Go's cgo LDFLAGS allowlist (see activation_policy_darwin.go).
export CGO_LDFLAGS_ALLOW='-Wl,-no_warn_duplicate_libraries'
wails build -platform "darwin/${ARCH}" -clean wails build -platform "darwin/${ARCH}" -clean
# 5. Re-sign with the stable dev cert. We invoke sign.sh through env var # 5. Re-sign with the stable dev cert. We invoke sign.sh through env var
+3 -1
View File
@@ -35,7 +35,9 @@ echo "================================================================"
# ---- 1. Build ---- # ---- 1. Build ----
echo "" echo ""
echo "[release.sh] (1/4) wails build" echo "[release.sh] (1/4) wails build"
( cd "${ROOT_DIR}" && wails build -platform "${WAILS_PLATFORM}" -clean ) # Allow the -Wl,-no_warn_duplicate_libraries flag declared in the darwin cgo
# files past Go's cgo LDFLAGS allowlist (see activation_policy_darwin.go).
( cd "${ROOT_DIR}" && CGO_LDFLAGS_ALLOW='-Wl,-no_warn_duplicate_libraries' wails build -platform "${WAILS_PLATFORM}" -clean )
# ---- 2. Sign .app ---- # ---- 2. Sign .app ----
echo "" echo ""