GVOD + Analytics: Star Treks \\///

vlcsnap-2014-02-19-17h20m22s182

vlcsnap-2014-02-19-17h16m19s57

This is a fan studies and media assemblage experiment, loosely associated with Professor Abigail De Kosnik’s Fan Data/Net Difference Project at the Berkeley Center for New Media. It uses technology associated with copyright verification and the surveillance state to desconstruct serial television into a hybrid media form.

The motivating question for this work is simple. How does one quantize serial television? Given a television episode,  such as the third episode of Star Trek, how can it be measured and then compared to other episodes of Star Trek? Can characters of the original Star Trek television series be compared to characters in different Star Trek universes and franchises, such as comparing Kirk/Spock in Star Trek to Janeway/Seven-of-Nine in Star Trek Voyager? Given a media text, how do you tag and score it? If you cannot score the whole text, can you score a character or characters? How do characters or elements of a media text become countable?

Media Texts:

Star Trek (The Original Series), aka TOS, 1966 to 1969. Episodes: 79. Running time each: 50 minutes. English subtitles from subscene for Seasons 1, 2, 3.

Star Trek Voyager, aka VOY, 1995 to 2001. Episodes: S03E26 to S07E26, ie #68 to #172, a total of 104. Running time each varies between 45 and 46 minutes.

Media Focus/Themes:

The pairs of Kirk/Spock in Star Trek the Original Series and Janeway/Seven of Nine in Star Trek Voyager will be compared in a media-analytic fashion.

A popular fanfic genre is called One True Pairing, aka OTP, which is a perceived or invented romantic relationship between two characters. One of the best known examples of OTP is the pair of Kirk and Spock on TOS. Indeed, fanfic involving Kirk and Spock is so popular to have its own nomenclature, and is called slash, or slash fic.

The pair of Janeway and Seven of Nine are comparable to Kirk and Spock as both the Janeway and Kirk characters are captains of space ships, and both the Seven of Nine and Spock characters are presented as “the other” to human characters: both the borg and vulcans are presented as otherworldly, non-human. The two pairs are different in other areas, the most obvious being gender: K/S is male, J/7 is female.

Some edit tapes for K/S can be found on YouTube for Seasons 1, 2, and 3. Some fanvids for J/7.

Open Questions:

This is a meta-vidding tool with an analytic overlay. It takes serial television shows and adds facial recognition to count face time and change the focus of viewing to specific character pairs instead of entire episodes. Developing the technology to answer these analytic questions, answering and understanding the answers, and formulating the next round of questions is the purpose of this project.

1. Should the method be the first 79 episodes that the character-pairs are together? How do you normalize the series and pairs?

Or minute-normalized, after the edits? The current times are:

TOS == 79 x 50 minutes == 3950 “character-pair” minutes total

VOY == 104 x 43 minutes == 4472 “character-pair” minutes total

2. Best method for facial recognition.

One idea is to use openframeworks, and incorporate an addon. Get FaceTracker library. See video explaining it. Get ofxFaceTracker addon for openframeworks.

Another is to use opencv directly.

OpenCv documentation main page.

Tutorial: Object detection with cascade classifiers.

User guide: Cascade Classifier Training.

Contrib/Experimental: Face Recognition with OpenCV. See the cv::FaceRecognizer class.

Many, many variants go into this. Some good links:

Samuel Molinari, People’s Control Over Their Own Image in
Photos on Social Networks, 2012-05-08

Aligning Faces in C++

Tutorial: OpenCV haartraining Naotoshi Seo

Notes on traincascades parameters

Recommended values for detecting

 

3

ffmpeg concat

LBP and Facial Recognition Example with Obama

Simple Face recognition using OpenCV, Etienne Membrives, The Pebibyte

 

IEEE Xplore: Face detection, pose estimation, landmark localization in the wild, 2012

Xiangxin Zhu, Ramanan, D

 

 

3. Measuring “character” and “character-pair” screen time. How is this related to the bechdel test? [2+ women, who talk to each other, about something besides a man] Can be this used to visualize it or flaws as currently conceived? What is bechdel version 2.0? [2+ women, who talk to each other, about something besides a man, or kids, or family] Can we use this tool to develop new forms?

4. How to auto-tag? How to populate the details of each scene in a tagged format? If original sources have subtitles, is there a way to dump the subs to SRT, and then populate the body of the wordpress with the transcript? Or, is there a way to use google’s transcription API’s to try and upload/subtitle/rip?

5. Can the netflix taxonomy be replicated? Given the patents, can some other organization scheme be devised?

Methodology:

0. Prerequisites

Software/hardware base is: Linux (Fedora 20) on Intel x86_64, using Nvidia graphics hardware and software. Ie, contemporary rendering and film production workstation.

Additional software is required on top of this base. For instance, g++ development environment, ffmpeg, opencv, openframeworks.080.

Make sure opencv is installed.

yum install -y opencv opencv-core opencv-devel opencv-python opencv-devel-docs

See OpenCV Configuration and Optimization Notes for more information about speeding up OpenCV on fedora.

1. Digitize selected episodes for processing with digital tools

Decrypt via makemkv. Compress to 3k constant rip with HandBrake.

Using 720p version of TOS in matroska media container. Downloaded SRT subtitles from fan sites. Media ends up being: 960×720, 24 frames a second.

2. Quantize each episode to a select number of frames.

Make sure ffmpeg is installed.

yum install -y ffmpeg ffmpeg-devel ffmpeg-libs

 

Sample math as follows. Assume a fifty minute show has 24 frames a second. That is:

50 minutes x 60 seconds in a minute x 24 frames a second == 72k total frames an episode.

Assuming a one-frame-a-second sample resolution gives 3k frames for the total set of frames in TOS episode one. Use ffmpeg to create a thumbnail image ever X seconds of video. And set to one image every second.

Via:

TMPDIR=tmp-1
mkdir $TMPDIR;
ffmpeg -i $1 -f image2 -vf fps=fps=1 ${TMPDIR}/out%4d.png;

3. Sort through frames and set aside twelve frames of Kirk faces, twelve frames of Spock faces.

This is used later, to train the facial recognition. Note: you definitely need hundreds and even thousands of positive samples for faces. In the case of faces you should consider all the race and age groups, emotions and perhaps beard styles.

For example, meet the Kirks.

And here are the Spocks.

In addition, this technique requires a negative set of images. These are images that are from the media source, but do not contain any of the faces that are going to be recognized. These are used to train the facial recognizer. Meet the non-K/S-faces.

4. Seed facial recognition with faces to recognize. Scan frames with facial recognition according to some input and expected result algorithm, and come up with edit lists that can be used to frames that are relevant to the character-pair.

Need either timecode or some other measure that can be dumped with an edit decision list or specific timecode marks. Some persistent data structure? Edits made.

5. Decompose episode into character-pair edit vids.

Use edit decision list or specific timecode marks, as above. Automate ffmpeg to make edits.

6. Store in wordpress container, one post per edit vid? Then with another post, tie together all of a single episode edit vids into one linked post?

Legal

There are both copyright risks and patent opportunities in this line of inquiry.

Production Notes:

Further
cinemetrics
How Netflix Reverse Engineered Hollywood, Alexis Madrigal, The Atlantic, 2014-01-02

Gone Awry Goes Kindle

ga_cover_wuxga_300s

Overview

Create an experimental electronic artist book for Kindle. Using the Manga/Comic/Graphic Novel example provided as part of Kindle Format 8 Publishing Tools to create the electronic version of Gone Awry.

Use KindleGen and Kindle Previewer to create the electronic edition and check quality. The linux version of calibre is used to check the generated .mobi file, and to tag and edit metadata. Then, upload the generated .mobi file to the Kindle Direct Publishing web-app, and fill in all rest of the metadata in a browser.

From the web app, publishing it on Amazon is easy.

KF8/Mobi

Amazon’s current proprietary format is: Kindle Format 8, or KF8. The file extension is .mobi. The Amazon specifications are in: Amazon Kindle Publishing Guidelines.

Parts of this also use OPF, the packaging for epub.

Target Display Hardware

Some key data points about display resolution of select current-model e-readers. Plan is to only support color, high-resolution devices.

  • Kindle Fire HD 7″ resolution is 1280 x 800 x 216 ppi8.9″ resolution is 1920 x 1200 x 254 ppi
  • Nexus Tablets 7″ resolution is 1920 x 1200 x 323 ppi (WUXGA, 1.6), 10″ resolution is 2560 x 1600 x 300 ppi (WQXGA, 1.6)
  • iPad 7″ resolution is 1024 x 768 x 163 ppi (XGA, 1.3), 9.7″ resolution is 2048 x 1536 x 264 ppi (QXGA, 1.3)

Versus layout for Gone Awry.

  • Print (19 x 13, 1.46)
  • ebook 1 (19.2 x 12, 1.6)
  • kindle publishing guidelines for fixed layout (1024 x 600 or 1536 x 900 zoom, 1.7)

Image Sizing and Aspect Ratio

First, resize existing .svg base documents in inkscape to the correct aspect ratio. For this project, the aspect ratio is 1.6. Thus, inkscape page sizes are changed from 19 x 13 inches to 19.2 x 12 inches.

This file is then exported as a transparent bitmap, at 2560 x 1600 @ 300dpi.

Next step is to process it in CS6 Photoshop on Mac OS X. The following transforms:

  • Layer->Flatten
  • Image->Image Size to 2560 x 1600
  • File->Save As jpeg, 9, 10, 11 qualities, but adjust to hit below 800k

HTML doc

Re-write example to suit Gone Awry, based on prototyping with a DocBook 5.0 XML form to generate the beginning HTML. Take this, edit the .opf and .ncx files to create a manifest and a table of contents/touch list, and bundle it up with the toplevel “images” directory such that the resulting filenames in the generated “gone-awry.opf” file are correct.

Kindle (KF8) doc

Unpack kindle binaries on linux, and run on the master list of files (the .opf file) as:

kindlegen gone-awry-v2.html -c0 -verbose gone-awry.opf

That should chug along, spewing a bunch of information. Pay attention, and fix any issues. The last bit of information before a successful creation of the .mobi file is related to the file size, and download time. Something like:

Info(prcgen):I1041: The file format version is V8
Info(prcgen):I1032: PRC built successfully
Info(prcgen):I15000:  Approximate Standard Mobi Deliverable file size :   0019466KB
Info(prcgen):I15001:  Approximate KF8 Deliverable file size :   0019526KB

From this, one can make back-of-the-envelope calculations about the royalty rates. For 70% royalties, the publisher must pay Amazon $0.15 per 1MB of .mobi file delivered. Thus, for a 40MB file, the delivery charge would be $6. So, the price would have to be more than this to break even. For 30% royalties, there is no delivery fee so pricing could feasibly drop below $6.

Result

The final book looks like “Gone Awry Kindle Edition” on Amazon’s marketplace.

Future work would be to shift to SVG containers, improving text legibility by un-rasterizing the text or presenting a vector text layer, and look at ways to add “scroll anchors” such that a “page flick” motion would speed through the next couple of pages before alighting on the specified anchor.

Background

Some notes from 2011 on self-publishing.

Current best sellers in the Kindle Store. August graphic novel sales, information, reviews.

Some of the meta-problems with pagination, and mapping electronic books to print books. Some meta-hating on some aspects of the current state-of-the-art.

A Digital Shift: Libraries, Ebooks and Beyond. Corey Doctorow, Library of Congress, 2013

Asama OG x Nezu Museum x Birgit Skiold

teahouse_stone_enso.6.1

teahouse_stone_enso.6.2

teahouse_stone_enso.6.0

teahouse_stone_enso.6.3

Protocol

Three visual allies cross space and time to meet for the first time at the Pacific Portal, 2013. Devastation Pacific, San Francisco, California, United States, Earth. At contact, self-identify as allies, murmur these are my people, make plans for a hybrid form.

COMBINE.

FUSE.

Set of four stone ensos with guilloche overlays, constructed via the lens of a slow walk through the Nezu Museum teahouse gardens, and committed to copper plates via direct gravure during Crown Point Press 2013 Summer Workshop.

Production Notes

Image Size
329 x 493mm (13 x 19 inches)

Plate Size
329 x 493mm (14 x 22 inches), .041in mirror polished copper plate

Finished Size
90mm borders x 4 sides means approximately 510 x 660mm (20 x 26in)

Plate 0: Zero
Scrap metal, found discard. Backside. Hand filed, uneven: “4eva.” Cancellation mark, “X,” top left of plate. Some Capp street sidewalk surfing scratches at two edges. Modified aquatint with surfer resin drops. Circle, brush, and drip spitbite.
Color: hottest pink || orange-y sepia || 20% prussian

  1. Pink Actual: Charbonnel Solferino Violet (~70%), Charbonnel Rouge Rubis/Ruby “Heart Stealer” Red (~30%), Douglas and Sturges Luster Pigment Pearl White, Douglas and Stuges Bronzing Powder Silvertone, Faust transparent base 2:1
  2. Orange Actual: Charbonnel Cardinal Red (~70%), Charbonnel Apricot Yellow, Douglas and Stuges Bronzing Powder Silvertone, Faust transparent base 2:1
  3. Blue Actual: Charbonnel Prussian Blue, white (~70%), Faust transparent base 1:1

Plate 1: One
One of 4 stone ensos. Direct gravure. Inkscape vector image composition with photo image collage. Photo tinting in photoshop to 2010 curves. Linux higest-resolution-possible-config to Epson 3880 (F18 adair create/F14 chula print configuration). Phoenix Gravure Pigment Paper.
Color: Black, silver, rust.
Color Actual: Charbonnel Black, Faust transparent base 1:1 to less.

Plate 2: Two
One of 3 deboss plates. Deboss/Emboss plates inspired by Birgit Skiold’s “Zen Gardens.” Copper plate, 0.041in scrap metal, discard. Flowjet cutting via TechShop, and the kind help of various mechanical engineering SF Techshop-ians! See Flowjet WaterJet aces, aka, CNC401: CNC Waterjet Cutter SBU – Level 4.

Paper
Somerset Velvet Soft White 300g
HM-63 Mohachi 300g (22 x 30″)
MMN-106 Torinoko Black 235g (38 x 74″)
Unknown Hiromi Dark Natural Gampi

The Master Technician
Etching consigliere Ianne Kjorli.

Links
Some printed pieces. Some background on the layering and combinations.

Ambient

Supreme court strikes down Doma on historic day for gay rights in America
SOMM, the new “Jiro Dreams of Sushi
Sans Soleil, Chris Marker
Royal Space Force/Wings of Honnêamise
Daft Punk, Duh
Lianne La Havas, No Room For Doubt
Joshua Redmond
Spring Breakers, crime art movie
Bling Ring, see above
Epic hating on pinterist
Mourning canceled shows that should have been on TNT, or made with animatronics.

Research Notes

Nezu Museum, Tokyo, Japan
Birgit Skiold, for those who don’t know Swedish pronunciation, Kathan Brown indicates this can be sounded out as “Beer-git Skee-old”
Blythe House Reading Room, Victoria and Albert Museum, London, UK. See the “Zen and the Art of Print: Birgit Skiold and Japan” show, and “Birgit Skiöld and the Print Workshop” for background on London’s first open-access print workshop.
Anish Kapoor, Untitled
Anish Kapoor, Untiled A

Conceptual Forms, Conceptual Cocktails

Le Classique, 2010

Photogravure, black ink on Somerset paper with plate mark and tone. 13 x 19″ (329 x 483mm).  Composition includes crop marks for A6 fill-in-the-blanks invitation, located at center-top of the plate.

This invitation reads:

CONCEPTUAL COCKTAIL PARTY

Yes there will be booze.

Details Marked Below.

HOURS

12   11   10   9   8   7   6   5   4   3   2   1

to

12   11   10   9   8   7   6   5   4   3   2   1

LOCATION

——————————————

On the roof ____

With projection _____

C++

San Francisco, CA

USA

Also present in the composition, are registration lines and fades. As part of the registration, centered on the text for the invitation, is the following production metadata:

PRINT THIS FOR FREE ON WHATEVER PRINTER WHEREVER YOU WANT!

Ask me how. Email printer@gnu.org and cc bkoz@gnu.org.

Underneath this in very small type there appears to be a mis-print, or type overlay. It is repeated, with some overprints, and says:

Then tell me how you did it!

—–

This is the first part of what was assumed to be a triptych composed of three etchings, all the same size. These were done half-way through the July 2010 Photogravure workshop at Crown Point Press, and etched and printed later in the summer and early autumn by Emily York. The workshop instructor was Emily York, with Asa Muir-Harmony. Attendees were Benjamin De Kosnik, Chayo de Chavez, Jay Dee Dearness, and Carolyn Dodds.

The three etchings are: Le Classique, Conceptual Cocktail, and a yet-unfinished piece that I call List Form. Plus one idea for another form, called SHOUT! All these engravings are arranged somehow on a wall, and also some kind of archival media that contains all the digital files used in the production of the plates is either attached to the back of the prints or otherwise incorporated into the display. In the summer of 2010, the archive consisted of a data DVD with outlines in Inkscape SVG files and TIFF files for photos, along with detailed system documentation for a linux print production workstation.  I did some cool drips on the DVDs with metallic ink to signify that they were ART, not DATA. That was what was attempted, at least. Now this part would probably be a folder on Dropbox and some links.

This is how I’d been looking at it for most of last year:

Le Classique, 2010

Conceptual Cocktail, 2010

List Form Plans, 2010

SHOUT! Idea, 2011

Production files for the third conceptual form, List Form, exist but have not yet been printed. This is in collaboration with Tomiyoshi Tsukada and is photogravure, guilloche, scanner-art with a three-layer silver overlay of text and fades.

There was another version of these prints conceptualized. It involved engraving a version of this with kif ink, presenting said version to John Gilmore in collaboration with Roland McGrath, and getting invited to James Turrell‘s Roden Crater. I’d like to think this version is still in development, but is admittedly a long shot.

These prints are very dear to me. But have largely defied any coherent explanation. Below is my latest attempt. Hopefully the passing of time has made it easier to explain what I was going for here.

This project is an investigation into printing forms and methodologies.

For the form experiments, I thought of printed matter, and tried to categorize them into distinct types or forms.

The first two etchings, Le Classique and Conceptual Cocktail, are examples of one of the most endearing and essential forms of printed communication, the invitation. This invitation is to an imaginary party. The third etching, List Form, is an example of another classic form of printed communication, the list. Grocery lists, to-do-lists, etc. The forth etching, SHOUT!, is an example of protest communication, the classic form for getting attention.

For the methodology experiments, I was trying to get at the essence of printed matter through the centuries, and the specialized knowledge and tools necessary to produce fine works on paper. Often, this knowledge is lost. Exact descriptions of some of the early photographic processes are lost forever. Hot type, ditched. Film pre-press, gone. Distressingly, digital files used in the production of works less than ten years old can become obsolete and rendered useless by proprietary format, abandoned products, and the inability to transfer/convert to newer production methods.

Influenced by the production methodologies of the free software movement, (started by Richard Stallman trying to freely communicate with a printer), I created a reference platform for the archiving of print production, based on the free/open source linux operating system and applicable tools, transforming proprietary curves used in the production of photogravure with free intellectual property. And a documentation system for production notes. Then, I used these files to create a reference print. The final step was to then attempt to dethrone this canonized static object, by inviting other participating printers to use these files to create this or altered works, and send me a copy, with documentation on how they got the print so that I could incorporate any new techniques.

Chayo Chavez indicated that this is a way to explain the free-software movement to art people, using their terminology. I considered this a pleasant by-product of the experiment.

As such, I expect this project to be ever-unfinished. But always worthwhile.

As Shown

Conceptual Cocktail, as above but in a swank frame, will be shown as part of “Traces, Marks, Fragments” at the San Luis Obispo Museum of Art (SLOMA) January 20th-February 26, 2012. This is a show juried by Sandow Birk.

Of Interest

Hiroshi Sugimoto, Conceptual Forms, as seen at Altalier Brancusi, Paris, 2006
Man Ray, Objet Mathematique, 1934-36
Edward Ruscha, Stains, 1971-75
Richard Stallman, The GNU Manifesto, 1985

Official Papers, Passport Gravure

Hand washes and american drips on paper for eventual monoprinting using the Pirate Passport Photogravure Plates and layers mapped out previously.

 

image

Yo, sharing. Get your papers, get legal. Sharing. Get your sharing papers.

image

image

Paper is Fabriano Rosaspina, Cream, 285 gsm. Each sheet is then cut into quarters of approximately 14″ by 19.5″. Plan is to print each of the four compositions, in each of the two color schemes, with three editions of each set. This gives a total of 4 x 2 x 3 = 24 pages.

Of the 24 pages, 12 pages have an additional element consisting of a brush stroke wash. These twelve pages are for monoprinting. There are three sets of 4 compositions. Pigments are suspended in a 14 baume gum arabic solution and applied by brush, by watercolor wash, and by dripping. Pigments used are Afghani Lazurite, Mayan Blue, and Diamond Dust.  See the color compositions for the Piracy Project Photogravure plates, with the inks as per the intaglio color scheme detailed below.

Plate orientation on the page:

For the monoprints, plate is centered horizontally, and approximately 4 inches from the top of the portrait-orientated sheet of paper.

For the rest of the prints, the plate is centered horizontally and vertically, approximately 6.5 inches from the top of the portrait-orientated sheet of paper.

Inks:

The charbonnel ink colors are as follows, all using Faust transparent base.

A: ocean blue, prussian blue, solferino violet or cardinal red or 70% gray or silver bronzing

B: primrose yellow, indian yellow, cardinal red or 70% gray or silver bronzing.