TensorFlow Configuration and Optimization Notes

Notes for installing TensorFlow on linux, with GPU enabled.

Background

TensorFlow is the second-generation ML framework from Google. (See this comparison of deep learning software.) The current state-of-the art image recognition models (inception-v3) use this framework.

Prequisites

Assuming Fedora 24 with Nvidia 1060 installed, running nvidia as opposed to nouveau drivers. See Fedora 24 Notes, and RPM Fusion’s installation page for installing the Nvidia drivers. In sum,

dnf install -y xorg-x11-drv-nvidia akmod-nvidia "kernel-devel-uname-r == $(uname -r)"
dnf install xorg-x11-drv-nvidia-cuda
dnf install vulkan

After, install some devel packages.

dnf install -y vulkan-devel

Download the Nvidia GPU CUDA Toolkit. The version used for this install is 8.0.61, and the network install for Fedora x86_64 was used.

This version of CUDA Toolkit is not C++11/C++14/C++17 aware. So, be aware! One way around this is to mod like below, and use -std=gnu++98.

117c117,118
 5
---
> /* bkoz use -std=c++98 if necessary */
> #if __GNUC__ > 6

Next, compile top-of-tree OpenCV (aka 3.2) with CUDA enabled. To do so, use the following configure list, mod for paths on system:

cmake -DVERBOSE=1 -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -std=gnu++98 -Wno-deprecated-gpu-targets" -D BUILD_EXAMPLES=1 -D BUILD_DOCS=1 -D WITH_OPENNI=1 -D WITH_CUDA=1 -D CUDA_FAST_MATH=1 -D WITH_CUBLAS=1 -D WITH_FFMPEG=1 -D WITH_EIGEN=1 -D ENABLE_FAST_MATH=1 -D ENABLE_SSE3=1 -D ENABLE_AVX=1 -D CMAKE_BUILD_TYPE=RELEASE -D ENABLE_PRECOMPILED_HEADERS=OFF  -D CMAKE_INSTALL_PREFIX=/home/bkoz/bin/H-opencv -D OPENCV_EXTRA_MODULES_PATH=/home/bkoz/src/opencv_contrib.git/modules /home/bkoz/src/opencv.git/

Admittedly, this abuse of CMAKE_CXX_FLAGS is not optimal. Maybe EXTRA_CXX_FLAGS?

Now, for Nvidia cuDNN. The version used for this install is 5.1

When that is done, use pip to install TensorFlow.

sudo pip install --upgrade pip;
sudo pip install tensorflow-gpu

This should output something like:

Collecting tensorflow-gpu
  Downloading tensorflow_gpu-0.12.1-cp27-cp27mu-manylinux1_x86_64.whl (89.7MB)
    100% |████████████████████████████████| 89.7MB 19kB/s 
Requirement already satisfied: mock>=2.0.0 in /usr/lib/python2.7/site-packages (from tensorflow-gpu)
Requirement already satisfied: six>=1.10.0 in /usr/lib/python2.7/site-packages (from tensorflow-gpu)
Requirement already satisfied: numpy>=1.11.0 in /usr/lib64/python2.7/site-packages (from tensorflow-gpu)
Collecting protobuf>=3.1.0 (from tensorflow-gpu)
  Downloading protobuf-3.2.0-cp27-cp27mu-manylinux1_x86_64.whl (5.6MB)
    100% |████████████████████████████████| 5.6MB 284kB/s 
Collecting wheel (from tensorflow-gpu)
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 3.3MB/s 
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow-gpu)
Requirement already satisfied: funcsigs>=1 in /usr/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow-gpu)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from protobuf>=3.1.0->tensorflow-gpu)
Installing collected packages: protobuf, wheel, tensorflow-gpu
Successfully installed protobuf-3.2.0 tensorflow-gpu-0.12.1 wheel-0.29.0

After this has completed, add in Keras.

Optimization

For Nvidia GPUs, take a look at this interesting post from Netflix. In sum, add

NVreg_CheckPCIConfigSpace=0

Fedora 25 Nvidia Notes

Step one: install Fedora 25.

Step two: turn off Wayland as the default. This is pretty simple, ie change /etc/gdm/custom.conf

WaylandEnable=false

When the GUI is started again, the following (assuming the current session is 2)

loginctl show-session 2 -p Type

Shows:

Type=x11

Then look at the RPM Fusion page for Nvidia.

Step three: Nvidia drivers are not working on Fedora 25. Instead, revert to Fedora 24 and proceed as above.

Notes on the Deep Deep Deepest

 

Reading

 

Approaches

  • SVM (Support Vector Machines)
  • RBM (Restricted Boltzmann Machines)
  • NN/Convolution NN/DNN

 

Silicon Valley Fun

TensorFlow Dev Summit

February 15, 2017 @ googleplex, Mountain View, CA

 

Software

theano

  • python-theano-doc
  • python3-theano
  • python2-theano

tensorflow

  • TensorFlow
  • github
  • Current models for facial recognition include VGG-19, VGG-16, and inception-v3. Of the listed models, inception-v3 seems to have the advantage, at least as of early 2017.

keras

scikit-learn

opencv

 

GPU Hardware

Recommended GPUS are: Nvidia GTX 1080, 1070, 980, and 970. Maximize CUDA cores.

Cartography Futures

1.dymaxion to authagraph to myriahedral projections
Buckminister Fuller and co-cartographer Shoji Sadao, designed an alternative projection map, called the Dymaxion map. Since then, several other projections have been proposed that evolve the idea of map projections in a similar manner. Favorites are:
2. new approaches to population density, use natural earth data and then read the shapefile with python

 

4. geospatial python
http://geospatialpython.com/

 

5. example api

Fedora 25 Notes

Install:
San Francisco, California USA

Using x86_64 netinstall images as install media.

Update/Post-Install:

Use RPM Fusion Repositories.

Some basics

dnf install -y tcsh gnome-tweak-tool livecd-tools ncftp lftp

Install KDE, Razor-qt, and MATE Desktops

	 	 
dnf install -y lightdm-gtk-greeter-settings mate-desktop mate-applets nm-connection-editor network-manager-applet caja mate-themes bluecurve-gtk-themes gnome-themes-legacy;
dnf groupinstall mate-desktop;  
dnf groupinstall kde-desktop;
dnf install -y gnome-themes;

Still a little sketchy, but optionally mate-compiz can be installed:

dnf groupinstall mate-compiz;

Install the basics for VLC

dnf install -y vlc vlc-core vlc-extras;
dnf install -y gaupol mediainfo

Install media xtra codecs.

dnf install -y ffmpeg ffmpeg-devel ffmpeg-compat-devel gstreamer-plugins-bad gstreamer-plugins-ugly gstreamer-plugins-good gstreamer-plugins-good-extras gstreamer-plugins-bad-free gstreamer-plugins-bad-free-extras gstreamer-plugins-bad-nonfree gstreamer-ffmpeg;
dnf install -y k3b-extras-freeworld;

Install negativo.
Use negativo17 repositories for HandBrake, volia.

dnf config-manager --add-repo=http://negativo17.org/repos/fedora-handbrake.repo;
dnf install -y HandBrake-gui HandBrake-cli;
dnf install -y makemkv mkvtoolnix mkvtoolnix-gui;
dnf install -y libdvdcss;

Use makemkv for Bluray.

Install some editors, utilities, and basics

dnf install -y abiword emacs tcsh

Install devel 1

dnf install -y openni openni-devel openni-doc  tbb tbb-devel tbb-doc eigen3-devel eigen3-doc openrdate simple-mtpfs emacs python-BeautifulSoup

Install devel 2

dnf groupinstall -y "Development Tools";
dnf install -y gcc-c++ libstdc++-devel libstdc++-docs boost boost-devel boost-doc valgrind gdb elfutils nemiver dwarves dejagnu emacs expect flex bison cmake git-svn mpfr-devel libmpc-devel lsb;
dnf debuginfo-install glibc;

Install database

dnf install -y glom libpqxx-devel libpqxx postgresql-devel

Install computer vision (hand-build clandmarks)

dnf install -y CImg-devel rapidxml-devel opencv opencv-devel

Install graphic

dnf install -y freetype-freeworld inkscape* ufraw ufraw-gimp 

Install technical documentation toolchain

dnf install -y pdfedit doxygen doxygen-latex graphviz  docbook5-style-xsl docbook5-schemas docbook2X docbook-style-xsl dblatex texinfo-tex dblatex docbook-utils-pdf  graphviz-python graphviz-R graphviz-graphs graphviz-doc R-core

Install sysadmin

sudo dnf install -y system-config-network system-config-samba system-config-services system-config-users system-config-firewall system-config-httpd

See separate posts on getting Nvidia and TensorFlow optimized CUDA up.

Issues

Something wrong with file indexing, it is consuming gigabytes of memory. Turn off search in the Gnome menu, but this is insufficient. Instead, use

balooctl disable

Set ulimits higher. Modify /etc/security/limits.cof like so:

*** limits.conf.f25	2016-12-11 23:27:46.609458649 -0800
--- limits.conf	2016-12-15 20:16:57.289245497 -0800
***************
*** 52,57 ****
--- 52,58 ----
  
  #*               soft    core            0
  #*               hard    rss             10000
+ 
  #@student        hard    nproc           20
  #@faculty        soft    nproc           20
  #@faculty        hard    nproc           50
***************
*** 59,61 ****
--- 60,67 ----
  #@student        -       maxlogins       4
  
  # End of file
+ 
+ bkoz               hard    nofile	8092
+ bkoz               soft    nofile       8092
+ 
+ 

In addition, it seems as if there is a bug with nmcli when using the Mate window manager. Mostly, it doesn’t work (along with nm-applet when used in a programmatic fashion from C/C++ code. So, use GNOME, as much as it hurts.

Ben Fry Geospatial

Processing started with a map of zip codes. Since then, Ben Fy has made some extraordinary maps of the United States, a few of which are listed below.

zipdecode
1. zipdecode. How does the postal code numbering work in the United States?

allstreets
2. allstreets.

Bittorrent Notes

Prerequisites

Assuming Fedora 24, with GeoIP installed as

dnf install -y GeoIP GeoIP-devel GeoIP-GeoLite-data GeoIP-GeoLite-data-extra;
dnf install -y libcurl libcurl-devel;
dnf install -y libevent libevent-devel --allowerasing;
dnf install -y intltool;

Test that this is working by using geoiplookup on the command line

%geoiplookup 8.8.8.8
GeoIP Country Edition: US, United States
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94035, 37.386002, -122.083801, 807, 650
GeoIP ASNum Edition: AS15169 Google Inc.

Other dependencies: rapidjson, rapidxml for visualization interface.

dnf install -y rapidjson rapidjson-devel rapidxml-devel;

Assuming SSL and crypto support.

dnf install -y openssl openssl-libs openssl-devel;

For the bittorrent protocol, use libtorrent built from source.

In addition to the operating system packages, python support for geolocation is needed. In particular, need bencode, geoip2, requests. So:

pip install GeoIP
pip install GeoIP2
pip install bencode
pip install requests

Background

Part one of Bittorrent distribution research was Fall 2015. Part two is Fall 2016.

Bittorrent. Common terms and jargon.

Basic idea as per The BitTorrent Protocol Specification. Of this, of note is tracker and in particular the tracker scraper protocol.

Magnet links wikipedia entry. PEX, DHT, Magnet links all from lifehacker.

Questions

1. Trackers. List of trackers, announcements, UDP, http.
Fetishizing the most current trackers: 1, 2.

2. Public torrents, Private torrents, SSL torrents. Why are some of these able to be scraped, and others, not so much? From transmission-show -s my.torrent, will get status for public and private, but not ssl torrents.

3. Look at transmission dependencies: openssl, libcurl. See libtransmission includes: transmission.h, variant.h, utils.h. See: struct tr_tracker_stat, tr_torrentTrackers, tr_torrentTrackersFree, tr_info, tr_tracker_info, tr_torrent_activity. Of note, it looks like multiple reads of peers from tracker. TR_PEER_FROM_PEX, TR_PEER_FROM_DHT, TR_PEER_FROM_TRACKER.

4. Look at SSL cert example in libtorrent. See libtorrent github source repository.

5. What kind of tree/node visualizations will work? See The Book of Trees.

Deliverables

One. Data set 1 is four torrents from October 11 for the same recent serial television episode, each a different uploading group: LOL, DIMENSION, AFG, mSD.

Two. Data set 2 is three torrents from October 17, over 8 locales, during 3 time periods.

Three: Prepare for TWD 2016-10-23 Season 7 premiere.

  • visualization: three or more approaches, data types, presentation/exhibition,prototype
  • scraper analytics
  • archival data format
  • persistent scraping, archival interface