sunglint

Protected: Mozilla Android 2024

2024/05/07

Filed under technology Tagged with android

Origin/URL Top List

2024/01/23

Research for browser and internet testing often involves running browsers in an automated way through many sites. This list of sites is called a “sitelist” and can be picked at random, or services like TP6 or Google’s CrUX can be used to generate sitelists.

TP6, and other surveys of the internet sites with specific characteristics often cost money. Research shows that using Google’s CrUX toplists can give as good or better coverage than many of the other proprietary offerings.

More info at the crux-top-lists github repo.

Here is the main entry point for CrUX on BigQuery.

Here’s the SQL for the top 100 from December, 2023:

SELECT distinct origin, experimental.popularity.rank

FROM `chrome-ux-report.experimental.global`

WHERE yyyymm = 202312

GROUP BY origin, experimental.popularity.rank

ORDER BY experimental.popularity.rank

LIMIT 100;

Several of these queries, for global and select countries, are saved in the CrUX 2024 sandbox on BigQuery.

Filed under Uncategorized

Protected: LoRA Notes

2023/12/21

Filed under art, research, technology Tagged with genai, lora

Deeper Still: Stable Diffusion Notes

2023/12/01

Run Stable Diffusion on linux, use Fedora 37 baseline.

Installation notes. Nothing to it.

Prompt creation is the creative zone right now.

Midjourney

See Nick St. Pierre (on twitter as nickfloats) for some prompt hacking for midjourney. His midjourney prompts look like:

cinematic, cowboy, hillside, high noon, gritty, grainy film negative, high contrast, burnt orange earth & periwinkle skies --ar 4:5 --style raw --stylize 420 --weird 69 --chaos 13 --turbo --repeat 20

Stable Diffusion

A couple of different UIs for this.

easy diffusion
automatic1111, then http://127.0.0.1:7860/

Basic models and key ideas, List of different custom stable diffusion models, including Hugging Face’s stable-diffusion-2, and sdxl. The AUTOMATIC1111 model page is perhaps the most useful, especially the parts about yaml config files for the 2x models.

Huggingface.co, prompt generator,

Prompt guide, fotor, beginners guide, art guide,

underwater temple, (ancient city of atlantis), high detail, majestic city, dramatic scale, 8k, blue, fish, (coral reef), Greek temple, Greek architecture, godrays, cinematic lighting, concept art, distinct, in the style of studio ghibli

Use negative prompts to remove unwanted things. So, if you want to remove “mustache” in the image, set the negative prompt to “mustache.”

Use (increase strength) and [reduce strength]

hyperrealistic

resolution = unreal engine, sharp focus, 8k, vray

dramatic, expansive, low angle, psychedelic

subject == scene description

medium == “digital painting”, “portrait”, “concept art”, “ultra realistic illustration”, underwater portrait, underwater steampunk

artist == by NAME

details chain with , DETAIL

Playground

for making prompts for multiple AI generators

RunwayML

web-ui for inpainting, etc.

civitai

Library of models and prompts

To make your own models, see this workflow post.

LLAMAFILE

here is a product from Moz that lets you use “executable” LLM files that launches a chat interface on a local webbrowser session to run jobs.

Filed under art, technology Tagged with midjourney, stability, stable diffusion

Notes on Large Format Laser

2023/10/26

Approximating the width of a copper plate means the canonical metal type is 18 gauge mirror-finish copper (aka 1.02mm), closest available acrylic that works is 1/16th inch (aka 1.5mm).

Vector cut lines are 100% R, and 0.025mm stroke (0.066mm works in practice, which is .18pt).

There are a variety of acrylics

1.5mm transparent clear gloss (2022)
1.5mm silver mirror with mirror side semi-matte (2023)
1.5mm frosted/matte
1.2mm silicone in grades medium to hard. ~~Get medium-soft~~. Get hard.

Remember, images (text especially) have to be reversed. (If the object is going to be displayed with the laser-cut side face-down).

Star-X v11, 2022

This was two horizontal halves of acrylic, a top half that was 22×30 and a bottom half that was 22×30, that when put together formed a 44×30 inch work. There is a vector cut overlay for the center star. And two 300 dpi PNG images that have not been flattened, with red vector lines representing vector cuts, and black lines part of it that would be rasterized.

fiber linesize: .50pt, 100% black line
satellite triangle radius: 4.6px, red cut line
tor exit nodes: .2pt, 55% black
ix: .188 pt, 100% black

Star-X Southern, v12-13, summer 2023

But there is also the art of getting a whole mess of grouped vector paths to print out out a laser printer. First, ungroup into. This attempt uses paths directly, instead of PNG groups. It is a layered SVG with each layer having some impact.

fiber linesize: .55pt 100% black line
satellite triangle radius: 4.2px, red cut line
tor exit nodes: .263pt, 25% black
ix: .63pt, 100% black
graticule inner: .35mm / 1pt, dashes pattern 13
graticule edge: .71mm / 2pt pattern 14

Antarctica imaged as 50% gray, which was not enough. Should this and the north pole/star be cut?

If this is to be a woodcut, then the base station triangles should not be cutouts. This is more problematic this year, as there are more ground-stations in the Americas, and if they are cut out then they will make the whole plate unstable. Should they be smaller?

Star-X v14.8, winter 2023

This is for laser etching (reverse) on the back of a mirrored acrylic (1.5mm) at 44 x 30 size. It’s broken into two halves vertically, a top half and a bottom half. These are tile-able with a 180 degree turn, and (a stylized) cabling route runs through the Pacific and Atlantic oceans.

fiber linesize: .2pt (.071mm), 100% black line
satellite triangle radius: 3.6px, 100% black fill
tor exit nodes: .263pt, 25% black
ix: .63pt, 100% black
graticule inner: .35mm / 1pt, pattern 16
graticule outer: .71mm / 2pt pattern 14

Star-X v14.9, valentine 2024

This is for laser etching (reverse) on the back of a silicone hard square (1.5mm) at 6×6 size.

fiber linesize: .55pt, 100% black line
satellite triangle radius: 3.6px, 100% black fill
tor exit nodes: .263pt, 25% black
ix: .63pt, 100% black
graticule inner: 0.75pt, pattern 12
graticule outer: .71mm / 2pt pattern 14

See setup hints from Pagoda Arts. Cut lines are styled as Red, 0.025mm stroke or smaller, aka 0.02.

Final production file format is PDF. Created via rasterizing vector layers (black/white) at 600 dpi, and then slotting raster layers in to new svg file as imported PNG. This is the production file SVG, with rasterized layers for cable, satellite, tor exit nodes, and ix. Cut layers are vector. Export the production file SVG raster and vector layers as a single pdf, run as a raster job on the laser cutters. Basically, this is pre-rasterizing the vector files on the beefy linux workstations so that what is presented to the laser imaging doesn’t blow them up with (linux-workstation run by madman) complexity.

NB: This is the thinnest vector line possible. On some angles and bezier curves, this width line may not cut through all of the mirror backing. Probably .25 is better, which means that inkjet resolution for visualization remains the highest-resolution.

NB: Maybe also vector grouping for drop-outs? Ungroup, convert? Or just hitting a machine complexity limit?

Filed under art, research, technology Tagged with acrylic, laser, laser cutting, laser etching, silicone, vector

robots.denai

2023/10/24

Background

Crawling and scraping networks have a long history that precedes the web. For example, modems connected to telephone networks scanned a range of numbers and marked which ones were fax machines. This mapping of the network is the essence of crawling: automated systems set up by humans that attempt to measure (aka “crawl” from the top of a list of addresses to the bottom) the web-based on some per-determined criteria. Scraping is just crawling and then saving the information.

Google has crawled the public web since 1998. Facebook does not allow scraping of its own sites, but contracts out scraping to others. The practice remains controversial. At the technical level, there is a tension between providing an API for structured data querying, and trying to restrict users who want to answer questions outside of the API or prefer the direct efficiency of scraping. Post scraping, the outputs are used to build indexes or summarize news headlines in ways that may violate copyright or terms of service.

While these legal disagreements continue to play out, on the technical side system administrators developed a solution (starting in 1994) using a simple file called robots.txt, and developed into a web standard called Robots Exclusion Protocol (IETF 9309). This is a file at the root level of a website that asks machines reading the site to please not read particular directories, or perhaps more sternly, asks machines not to read the site at all. A better definition of machines would be computers running software to copy or detail the contents and structure of the website for analysis by search engines, or as use as inputs for generative ai, or for any of a million reasons. Robots, say. Scrapers, say. People want to know more about the internet, or a particular site.

The contents of this file are in public, and don’t need logins or any special access for humans or robots to read.

Example robots.txt files

Example Generative Scraper Opt-Outs

Open AI, GPTBot, since 2023-08-07

CCBot, Common Crawl, List of Naughty Obscene… Words

Protected: Moz Browsertime (desktop linux a11y test host)

2023/09/29

Filed under technology

Fedora 38 Notes

2023/09/28

Install:
San Francisco, California USA

Using x86_64 netinstall images as install media, workstation iso to usb.

Update/Post-Install:

Some basics

dnf install -y tcsh fish emacs livecd-tools nload gparted util-linux-user at
dnf install -y nfsv4-client-utils nfs4-acl-tools nfs-utils-coreos nfs-utils

Install for GNOME, and tweaks

sudo dnf install -y gnome-tweaks gnome-shell-extension-common gnome-shell gnome-shell-extension-user-theme gnome-shell-extension-dash-to-dock gnome-shell-extension-apps-menu gnome-shell-extension-places-menu

Install KDE, and the various GNOME 2ish desktops aka MATE and Cinnamon Desktops

dnf groupinstall "GNOME Desktop Environment" "KDE Plasma Workspaces" "MATE Desktop" "Cinnamon Desktop"

Install tailscale

sudo systemctl enable --now tailscaled sshd;

sudo tailscale up

Set default groups, users, limits.conf settings. Here’s how to set up your ssh keys.

Use RPM Fusion Repositories.

Install the basics for media and VLC

dnf install -y vlc vlc-core vlc-extras;
dnf install -y gaupol mediainfo

Install media xtra codecs.

dnf install -y ffmpeg ffmpeg-devel gstreamer-plugins-bad gstreamer-plugins-ugly gstreamer-plugins-bad-nonfree gstreamer-ffmpeg;

Install negativo.
Use negativo17 repositories for HandBrake or MakeMKV, volia. Use these directions for MakeMKV.

dnf config-manager --add-repo=http://negativo17.org/repos/fedora-handbrake.repo;
dnf install -y qt5-qtbase qt5-qtbase-devel
dnf install -y HandBrake-gui HandBrake-cli;
dnf install -y makemkv mkvtoolnix mkvtoolnix-gui;
dnf install -y libdvdcss;

Use makemkv for Bluray.

Install some editors, utilities, and basics

Install devel 1

dnf install -y openni openni-devel openni-doc  tbb tbb-devel tbb-doc eigen3-devel eigen3-doc openrdate

Install devel 2

dnf groupinstall -y "Development Tools";
dnf install -y python3-beautifulsoup4
dnf install -y gcc-c++ libstdc++-devel libstdc++-docs boost boost-devel boost-doc valgrind gdb elfutils nemiver dwarves dejagnu emacs expect flex bison cmake git-svn mpfr-devel libmpc-devel lsb cmake ninja-build;
dnf debuginfo-install glibc;

Install database

dnf install -y glom libpqxx-devel libpqxx postgresql-devel

Install computer vision (hand-build clandmarks and opencv if optimized)

dnf install -y CImg-devel rapidxml-devel rapidjson-devel jq opencv opencv-devel

Install graphic

dnf install -y freetype-freeworld inkscape* ufraw ufraw-gimp fuse-exfat

Install technical documentation toolchain

dnf install -y pdfedit texlive-pdfjam doxygen doxygen-latex graphviz  docbook5-style-xsl docbook5-schemas docbook2X docbook-style-xsl dblatex texinfo-tex dblatex docbook-utils-pdf  graphviz-python graphviz-R graphviz-graphs graphviz-doc R-core

dnf install -y pdfarranger ghostscript ghostscript-tools-printing ghostscript-tools-fonts

See separate posts on getting Nvidia and TensorFlow optimized CUDA up.

Issues

Set ulimits higher. Modify /etc/security/limits.cof like so:

*** limits.conf.f25	2016-12-11 23:27:46.609458649 -0800
--- limits.conf	2016-12-15 20:16:57.289245497 -0800
***************
*** 52,57 ****
--- 52,58 ----
  
  #*               soft    core            0
  #*               hard    rss             10000
+ 
  #@student        hard    nproc           20
  #@faculty        soft    nproc           20
  #@faculty        hard    nproc           50
***************
*** 59,61 ****
--- 60,67 ----
  #@student        -       maxlogins       4
  
  # End of file
+ 
+ bkoz               hard    nofile	64000
+ bkoz               soft    nofile       64000
+ 
+

Filed under technology Tagged with 38, fedora, linux

Python 3.10/3.11 Side-by-Side Notes

2023/07/24

F38 defaults to Python 3.11.4, so the “python” and the “python3.11” commands are the same.

To use Python 3.10.x on this system:

sudo dnf install -y python3.10-libs python3.10-debug python3.10-devel python3.10

and then

python3.10

Launches a Python 3.10.12 shell.

To use a virtual environment set to this python version,

mkdir moz-python

(make sure you are in the BASH shell, say by typing the command “bash“)

python3.10 -m venv moz-python

source moz-python/bin/activate

Filed under technology Tagged with python, runtime versioning

GeoIP2 expanded fields: continents and country registered

2023/07/17

Base platform is fedora 38

sudo dnf install -y libmaxminddb libmaxminddb-devel

sudo dnf install -y geolite2-city geolite2-country geolite2-asn

download geolite2++, unpack

ccmake . to configure

make

Here are the various APIs to GeoIP2. And some idea of the continental slices. Background IP Geolocation Data for schema fields and metadata.

For the C++ API, use GeoLite2++ and here are the docs. Just use the pre-built binary from rizal.

To purchase a one-time GeoIP2 City-level database in binary format from MaxMind, go here. Stash the updated files in /usr/share/GeoIP.

The source interface is in the alpha60 file “a60-geo-maxmind-2.h”, and uses the following databases directly:

“/usr/share/GeoIP/GeoLite2-Country.mmdb”

“/usr/share/GeoIP/GeoLite2-City.mmdb”

“/usr/share/GeoIP/GeoLite2-ASN.mmdb”

Filed under research, technology Tagged with c++, geo, ip, linux

Mermaid x Markdown

2023/03/31

Doxygen integration, maybe?

But Markdown with Mermaid, structured in pages, is perfect and an easy way to make code flow diagrams.

Some Mermaid basics:

This gets translated into markdown like:

  ```mermaid
graph TD;
      A-->B;
      A-->C;
      B-->D;
      C-->D;
```

Here’s an example for doxygen.

Filed under technology Tagged with documentation, mermaid

Alma 9 Notes

2023/03/08

groupadd -g 10000 mutualists

useradd -G mutualists bkoz

usermod -aG wheel bkoz

dnf install -y tcsh emacs

hostnamectl set-hostname "yanamekiddo"

passwd bkoz

generate ssh keys

ssh-keygen -t ed25519

RPMFusion

wget https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-9.noarch.rpm

wget https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-9.noarch.rpm

dnf localinstall -y rpmfusion-*

 dnf --enablerepo=crb install ninja-build cmake

dnf install -y rapidjson-devel boost boost-devel

dnf install -y vlc vlc-core vlc-extras gaupol mediainfo

dnf groupinstall -y "Development Tools";

edit limits.conf

bkoz hard nofile 64000
bkoz soft nofile 64000

dnf install -y nload NetworkManager NetworkManager-openvpn openvpn
dnf install -y ffmpeg ffmpeg-devel;
dnf install -y openssl openssl-libs openssl-devel jq
dnf install -y inkscape ghostscript

timedatectl set-timezone Europe/Berlin

yum install -y at
systemctl enable atd.service
systemctl start atd.service

dnf install -y util-linux-user


Post-Install

filesystem rescue

tailscale

cockpit

Filed under Uncategorized

Protected: Speedometer 3

2023/03/06

Filed under Uncategorized Tagged with technology, web

Internet Cartography

2023/01/30

Filed under Uncategorized

chrome/chromium CLI

2022/11/21

The list of chrome CLI settings

Filed under technology Tagged with chrome, web automation

Protected: Influx/Grafana Docker Setup

2022/10/04

Filed under technology Tagged with graphana, influx, influx v2

Fedora 36 KVM for Windows 11 Guest + WSL2 + Fedora 36

2022/09/03

Round and round and round…

So, not as straightforward as expected. Are you used to stupid-easy KVM installs of Windows 10? That’s not the way of Windows 11.

Start with virtualization environment on current Fedora linux.

 sudo dnf install @virtualization

First, get the Windows 11 iso from Microsoft. Second, get a Windows 11 Pro Key.

See this post for some required steps and setup. For Fedora 36, that is

dnf install -y swtpm swtpm-libs swtpm-tools edk2-ovmf

And then the install is CLI via the slightly modified above:

virt-install --name win11 --ram 16144 --disk path=/home/bkoz/bin/kvm/windows_11.88GB.img,size=88 --cpu host-passthrough --vcpus=4 --os-variant=win10 --network bridge=virbr0 --cdrom /home/bkoz/mounts/auteuil/software/microsoft/windows_11/Win11_English_x64v1.iso --features kvm_hidden=on,smm=on --tpm backend.type=emulator,backend.version=2.0,model=tpm-tis --boot loader=/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd,loader_ro=yes,loader_type=pflash,nvram_template=/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd --check path_in_use=off

So, the preliminary KVM image should be built after this install. Next, optimize it.

To achieve performance, use the virtIO drivers (latest). Workflow is, download the latest .iso file (virtio-win-0.1.225.iso), in virt-manager go to machine details and add a new device, which is a SATA CD-ROM that connects that ISO file to the virtual machine. Boot up the virtual machine with that iso image attached, and then select it in the Windows UI, and look for the application “virtio-win-guest-tools” and double click it, and install all components. Restart the virtual machine with the ISO file removed.

See virtio-win-pkg-scripts.

After the install, put Linux on Windows (WSL) on it (get a feel for the steps with this video, some basic commands). First, search for “Windows Features” and enable “Hyper-V.” Then, search for “Command Prompt” and then right-click the application that comes up to open it in Administrator mode. When the terminal opens, type

wsl --install

To install Fedora 36, download the base image from koji, see Fedora-Container-Base packages, get an archive file and extract the “layer.tar” file from it with the root filesystem. Install this in the windows guest by opening a power shell and (first make \WSL and \WSL\Fedora_36 directories) and then typing

wsl --import Fedora_36 c:\WSL\Fedora_36 c:\Download\fedora-36-rootfs.tar

Then it can be installed

wsl --install -d Fedora_36

And run as root with

wsl -d Fedora_36

Microsoft suggests using WSL with Windows Terminal.

For video VGA use QXL, see spice-guest-tools. Maybe use spicey or virt-managers’s virt-viewer from spice-guest-tools.

To build Firefox on Windows, see the Firefox Source Docs.

Filed under Uncategorized

Leaks Attack!

2022/07/23

HOPE 2022, Using Topic Models to Organize Huge Leaks, 2022-07-23

HOPE 2022, Leaks and Hacks: Four Years of DDoSecrets, (youtube), 2022-07-23

DEFCON 2022, Leak the Planet, Emma Best & Xan North, 2022-08-12

DEFCON 2022, Computer Hacks in the RUS-UKR War, Kenneth Geers, 2022-08

Bodó, Balázs, The Genesis of Library Genesis: The Birth of a Global Scholarly Shadow Library, Shadow Libraries, MIT Press, 2018

De Kosnik, Abigail, Piracy is the Future of Culture: Speculating about Media Preservation after Collapse, Third Text, 2019

De Kosnik, Benjamin and De Kosnik, Abigail. Network Activity Monitoring Service, US10911337B1

Ensafai, Roya, A look at router geolocation in public and commercial databases, 2017, IMC

Guerrero-Saade, Juan Andrés. Hacktivism and State-Sponsored Knock-Offs | Attributing Deceptive Hack-and-Leak Operations, 2022

Li, Jinying, Pirate cosmopolitanism and the undercurrents of flow, Transnational Convergence of East Asian Pop Culture, Routledge, 2021

McLaughlin, Jenna. How a nonprofit group has become the biggest repository for hacked Russian data, NPR, 2022-07-05

Karaganis, Joe, Access from Above, Access from Below, Shadow Libraries, MIT Press, 2018

Karaganis, Joe and Renkema, Lennart, Copy Culture in the US & Germany, 2013

Philip, Kavita. The Internet Is A Leaky Pipe Made of Imperial Rubble, Your Computer Is On Fire!, MIT Press, 2019

Filed under Uncategorized

Mueller (Report + Manafort Response) Redactions Printed With Invisible Ink

2022/06/28

In 2019, a heavily-redacted version of the Mueller Report was made public. On the thirteenth of May this year, another heavily-redacted addendum was made public as part of the Paul Manafort sentencing, the Mueller Manafort Response.

If you take both of these documents, add them together, and shrunk every page down such that 486 pages fit on a 44 x 30 inch sheet of paper, it’d look like this. Reading left to right, row one to two, etc.

That big black box on the third to last row is what one page looks like, a pseudo registration mark. It’s a document break between the report and the response documents.

On visual inspection, of immediate note is the sheer number of black bars. What shade of gray is this? Maybe 20%, 40%? The Department of Justice very liberally blacked out and censored any text that it, on review, would do HARM TO ONGOING MATTER (often abbreviated as H.O.M), or PERSONNEL PRIVACY, among other reasons. These redactions make key parts of this document completely incomprehensible, especially anything to do with Russia’s Internet Research Agency and past and present activities on USA-based Social Media Platforms in and after the 2016 US Presidential Election.

If you then removed all the text from those aggregated pages, and just looked at redaction blocks, it’d look like this:

It takes a village folks.

Profound thanks to

Alex Thompson at Pagoda Arts, Emily York and Courtney Sennish at Crown Point Press, Kate Randall (invisible ink, burn art history, burn safety), Ian Roxbourough (combustion)

We stand on the shoulders of giants

Agnes Martin, John Cage(1, 2), Nam June Paik, Birgit Skiöld, Nance O’Banion, Jenny Holzer, Cai Guo-Qiang

Filed under activism, art, research Tagged with 2022, bullshit, california, crown point press workshop, emboss, FOIA, generative art, hand emboss, hey-bob-i-can't-read-this, internet-how-to-smoke-paper, internet-research-agency, invisible ink, laser cutting, manafort, manafort response, mueller, mueller report, redaction, redaction aesthetics, robert mueller, russia, san francisco, singed paper, smoke, sunlight, trump impeachment, ukraine

Docker Notes

2022/06/28

Cheat Sheet:

docker network ls // list networks
docker image ls // list images

docker image rm [image name]
docker rmi [image id] // docker rmi fd484f19954f
docker rmi [repository]:[tag] // docker rmi fd484f19954f

docker ps -a // list containers (running and stopped)
docker start [container name] // start
docker stop [container name] // stop
docker rm [container name]
docker logs [container name] // container logs

docker run (-d) [image_name]:[tag] --name upick // -d == detached

docker logs devpac_grafana  // view logs for thedevpac_grafana container

CLI hack: one can jump from the host to the docker image by mounting points on the host filesystem as bind mounts on the image (via --volume or -v flags). And also one can jump into a shell on the image via

docker exec -it devpac_grafana /bin/bash
docker exec --user root -it devpac_grafana /bin/bash


docker pull grafana/grafana:10.2.3

Filed under Uncategorized

← Older posts

sunglint

Protected: Mozilla Android 2024

Origin/URL Top List

Protected: LoRA Notes

Deeper Still: Stable Diffusion Notes

Notes on Large Format Laser

robots.denai

Protected: Moz Browsertime (desktop linux a11y test host)

Fedora 38 Notes

Python 3.10/3.11 Side-by-Side Notes

GeoIP2 expanded fields: continents and country registered

Mermaid x Markdown

Alma 9 Notes

Protected: Speedometer 3

Internet Cartography

chrome/chromium CLI

Protected: Influx/Grafana Docker Setup

Fedora 36 KVM for Windows 11 Guest + WSL2 + Fedora 36

Leaks Attack!

Mueller (Report + Manafort Response) Redactions Printed With Invisible Ink

Docker Notes

Tags

Archives