Live data from Hacker News

Self-updating screenshots

interblah.net

71–80 of 80 posts

Re: Self-updating screenshots

#71
post #55

The only problem with this idea I can forsee is that the application and therefore the screenshots can change but the documentation does not. For example, if the documentation says press "Options > Customize" but the application is updated so this becomes "Preferences > Advanced" then the screenshot will show the new text but the documentation will still show the old labels. This would be very confusing as it would b…

As a next step, it could be cool to write unit tests against these screenshots that look for words like you mentioned. That way if a screenshot is updated and a test breaks you will know what documentation to update

Re: Self-updating screenshots

#72
post #61
post #56

Speaking of screenshots. Can we please agree that the OS should not send any event to applications while a screenshot is being made? It is very annoying if you press a screenshot button and suddenly menus disappear. Or much worse, the application sends a "screenshot taken" message back to the social media platform.

I also can't stand Android preventing me from taking a screenshot. It's on my screen, I have the right to take a screenshot. I understand the technical limit of taking screenshots of DRM-protected content (e.g. Netflix), but why would my bank app be allowed to stop me from taking screenshots?

You can mod the APK and remove those detections using ReVanced Manager

https://github.com/revanced/revanced-manager

https://gitlab.com/ReVanced/revanced-patches/-/blob/main/ext...

Re: Self-updating screenshots

#73
post #26

Earlier quoted context omitted.

> It takes no time to include all this In some cases it does. Which engine?

I don't use an engine for vibing, they tend to be built around managing content and using their editors. You can drive engines from code but they are more built for invoking code from content usually. So I just use frameworks like SDL, Raylib C# and ebitengine. Most stuff I've done recently was ebitengine because I felt golang was the best thing to have LLMs writing when I started them. Right now I have my own framew…

Actually, if I were just getting started I might pick an engine at this point. It's probably great at Godot now, and maybe with MCP servers can work on any other engine fine by having access to the editor.

Re: Self-updating screenshots

#74

This is very useful in mobile projects. App stores require screenshots, but generating N images for NUMBER_OF_SCREEN_SIZES times NUMBER_OF_LOCALIZATIONS can be a chore. In the past I wrote my own scripts for that, today tools like Fastlane[1] help. I use Fastlane for my logic puzzle game Nonoverse[2], you can see sample screenshots in its App Store page. I also automated App Preview video recording, complete with mul…

These days it can be much easier to(though costlier) to use an agent skill.

Re: Self-updating screenshots

#75
post #70

Earlier quoted context omitted.

Solution: don't use mobile bank apps.

I'm forced to use a bank app in order to authenticate, even if I want to login on the desktop website. I think it's because of an EU regulation for strong authentication.

The implementation causing you issues is from the bank, not from the EU regulation.

See https://en.wikipedia.org/wiki/Strong_customer_authentication for details

Re: Self-updating screenshots

#76
post #53

Very cool. For the small casual games I've been vibe coding, I always start from a place where the application has a CLI where it can run headless, rendering to offscreen texture, with a a screenshot command as well as performance instrumentation. It takes no time to include all this, and gives the agent a way to automate the ui and inspect important things. It also lets me trivially have the agent update screenshots…

I was toying with a DragonRuby game a while back, and did something like that. But DR also comes with recording reproducible playbacks, screenshotting etc. built in, couple hot reloading and easily being able to inject code into the running game, and it was great putting in place instructions so the agent could run the game fully and show off things for me in addition to allowing it to test things. I think we'll see…

This is also what I've done for my multiplayer falling sand game: it's very much not vibe coded (too performance-sensitive), but coding agents can launch the game on my steamdeck and run benchmarks, take captures & verify rendering is bit-for-bit identical on a given machine, etc.

Agent can't _play_ the game yet, but that's on my list to experiment with.

Re: Self-updating screenshots

#77

Same, I've added a .#screenshots derivation. High up-front effort but almost zero maintenance afterwards. Bonus: since you're generating screenshots programmatically anyway, you can generate a pair of each with your app's light/dark theme, and swap them in/out depending on prefers-color-scheme: dark. elements work in GitHub READMEs, too: https://github.com/CyberShadow/CyDo#readme

+1 for this approach. For a mobile app, I made Nix spawn an ephemeral Android emulator instance for generating up-to-date screenshots, requiring no prior setup and leaving no lingering data around after running. Setting it up wasn't that high-effort in my case either; coming up with the idea was the hard part, the Nix code was one-shot by your favorite LLM. Granted manually updating the screenshots isn't the most lab…

That sounds so cool! Is the repo available anywhere?

Re: Self-updating screenshots

#79

Earlier quoted context omitted.

+1 for this approach. For a mobile app, I made Nix spawn an ephemeral Android emulator instance for generating up-to-date screenshots, requiring no prior setup and leaving no lingering data around after running. Setting it up wasn't that high-effort in my case either; coming up with the idea was the hard part, the Nix code was one-shot by your favorite LLM. Granted manually updating the screenshots isn't the most lab…

That sounds so cool! Is the repo available anywhere?

Nothing public yet, but this is the Nix output for taking the screenshot, to be executed via `nix run .#screenshot`:

        outputs.apps.x86_64.screenshot = {
          type = "app";
          program = toString (pkgs.writeShellScript "screenshot-script" ''
            set -euo pipefail

            EMU_SDK="${androidEmulatorComposition.androidsdk}/libexec/android-sdk"
            ADB="$EMU_SDK/platform-tools/adb"
            EMULATOR="$EMU_SDK/emulator/emulator"
            APK="${self.packages.${system}.debug}/myapp-debug.apk"

            SRC_DIR="$(${pkgs.git}/bin/git rev-parse --show-toplevel)"
            AVD_HOME="$(mktemp -d)"
            trap 'kill "$EMU_PID" 2>/dev/null; wait "$EMU_PID" 2>/dev/null; rm -rf "$AVD_HOME"' EXIT

            # Create AVD
            AVD_DIR="$AVD_HOME/screenshot.avd"
            mkdir -p "$AVD_DIR"
            cat > "$AVD_HOME/screenshot.ini"  "$AVD_DIR/config.ini"  Starting emulator..."
            ANDROID_AVD_HOME="$AVD_HOME" ANDROID_HOME="$EMU_SDK" \
              "$EMULATOR" -avd screenshot -no-window -no-audio -no-boot-anim \
              -gpu swiftshader_indirect -no-snapshot 2>&1 &
            EMU_PID=$!

            echo "==> Waiting for boot..."
            for i in $(seq 1 90); do
              BOOT=$("$ADB" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r') || true
              if [ "$BOOT" = "1" ]; then
                echo "    Booted after ~$((i * 2))s"
                break
              fi
              sleep 2
            done
            if [ "$BOOT" != "1" ]; then
              echo "ERROR: Emulator failed to boot" >&2
              exit 1
            fi

            # Enable dark mode
            "$ADB" shell cmd uimode night yes

            # Install and launch
            echo "==> Installing APK..."
            "$ADB" install -r "$APK"
            "$ADB" shell pm grant com.me.myapp android.permission.WRITE_SECURE_SETTINGS
            "$ADB" shell am start -n com.me.myapp/.MainActivity
            sleep 3

            # Navigate to settings screen by tapping "Notification Filters" button
            # This uses uiautomator to find the button by text for robustness
            "$ADB" shell uiautomator dump /sdcard/ui.xml
            BOUNDS=$("$ADB" shell cat /sdcard/ui.xml \
              | ${pkgs.gnugrep}/bin/grep -oP 'text="Notification Filters"[^>]*bounds="\K[^"]+' \
              || true)
            if [ -z "$BOUNDS" ]; then
              echo "ERROR: Could not find Notification Filters button" >&2
              exit 1
            fi
            # Parse bounds "[x1,y1][x2,y2]" to compute center tap coordinates
            X1=$(echo "$BOUNDS" | ${pkgs.gnused}/bin/sed 's/\[\([0-9]*\),\([0-9]*\)\]\[\([0-9]*\),\([0-9]*\)\]/\1/')
            Y1=$(echo "$BOUNDS" | ${pkgs.gnused}/bin/sed 's/\[\([0-9]*\),\([0-9]*\)\]\[\([0-9]*\),\([0-9]*\)\]/\2/')
            X2=$(echo "$BOUNDS" | ${pkgs.gnused}/bin/sed 's/\[\([0-9]*\),\([0-9]*\)\]\[\([0-9]*\),\([0-9]*\)\]/\3/')
            Y2=$(echo "$BOUNDS" | ${pkgs.gnused}/bin/sed 's/\[\([0-9]*\),\([0-9]*\)\]\[\([0-9]*\),\([0-9]*\)\]/\4/')
            TAP_X=$(( (X1 + X2) / 2 ))
            TAP_Y=$(( (Y1 + Y2) / 2 ))
            "$ADB" shell input tap "$TAP_X" "$TAP_Y"
            sleep 2

            # Capture and process screenshot
            echo "==> Capturing screenshot..."
            "$ADB" shell screencap -p /sdcard/screenshot.png
            "$ADB" pull /sdcard/screenshot.png "$AVD_HOME/raw.png"

            # Crop to content: remove status bar (top 128px) and empty space below
            # Per-App Overrides, then resize with high-quality Lanczos filter
            ${pkgs.imagemagick}/bin/magick "$AVD_HOME/raw.png" \
              -crop 1080x1100+0+128 +repage \
              -filter Lanczos -resize 540x \
              "$SRC_DIR/fastlane/metadata/android/en-US/images/phoneScreenshots/settings.png"

            echo "==> Screenshot saved to fastlane/metadata/android/en-US/images/phoneScreenshots/settings.png"
          '');
        };
Post reply on HN