Dell Precision 5820 Tower Notes

Notes for running Fedora 29 on the Dell Precision 5820 Tower workstation. The particular model has Intel Xeon W-2155, ie 10 cores at 3.3-4.5 GHz, 32GB RAM, 500GB NVMe drive, Nvidia P400 graphics cards with mDP outs.

BIOS Settings

Linux needs certain BIOS settings flipped:

  • Secure Boot Disabled
  • System Config -> SATA to AHCI (RAID might work but untested)
  • Misc Devices -> Enable SD Card, SD Card boot
  • Boot Sequence -> UEFI, UEFI boot path security off

Install

Used the netinstall ISO image written to a SD card, and then booted the computer off this device (Hold down the function 12 key at startup to get a boot menu.)

There is no getting around the video oddness when installing. Embrace the suck. This is the only tricky part of installing on this hardware (well, the SSD drive can be a bit problematic if not using a very current Fedora version, so don’t do that).

What worked:

Add

inst.text

to the linux boot line, which boots you into text mode (aka init 3). Then,
start the network and install the Nvidia drivers from RPMFusion.

ifup en01;
dnf install -y kmod-nvidia

This should add the nouveau modules to the blacklist on future boots, but to make sure this is how to do it on the linux boot line:

rd.driver.blacklist=nouveau modprobe.blacklist=nouveau

 

Dual Boot, Windows VM, etc.

Let’s be able to run windows 10, either via virtualization or natively.

To extract the windows key from the machine when using linux, try:

sudo cat /sys/firmware/acpi/tables/MSDM | tail -c 32 | xargs -0 echo

Then use that key to install windows 10 from the ISO, or use it in an existing KVM image.

To install natively, shrink your windows install away to a mere sliver so that most space can be used by a linux install via…

Just right click on “My Computer”, choose “Manage”, and you will get a popup window, choose “Disk Management” in the left navigation bar. And then, your hard disk condition will show as follow. Then you need to right click on the partition which you want to shrink, and choose “Shrink Volume”.

ARP Database Notes

Software configuration for commodity x86/86_64 linux art research platform.

Prerequisites

Assume Fedora 27. See base document for full software stack.

Database Setup 1: server mysql

Both the wiki and wordpress require a database. So, pick the MySQL database, which is packaged as mariadb on Fedora. Project page for MariaDB.

dnf install -y mariadb mariadb-server php-mysqlnd phpMyAdmin

Secure it (and set the root password on the database) with

mysql_secure_installation

And enable and start it via

systemctl enable mariadb.service
systemctl start mariadb.service

To use the web-ui to administer, go to

http://localhost/phpmyadmin/

Configuration file is: /etc/my.cnf
Log file is:/var/log/mariadb/mariadb.log
The site files are in prefix:/var/lib/mysql

Database Setup 2: per-user postgressql

For non-platform use, aka for individual users, pick the glom front-end for the postgres database. Some platform notes are here.

dnf install -y postgresql postgresql-server postgresql-libs postgresql-contrib phpPgAdmin

Initialize the database

postgresql-setup --initdb --unit postgresql

Then enable and start

systemctl enable postgresql
systemctl start postgresql

To use the web-ui to administer, go to

http://localhost:8080/phppgadmin

There is a graphical database called glom that is similar to FileMakerPro on the mac and Access on windows.

dnf install -y glom

Artistic Research Platform Notes

Software configuration for commodity x86/86_64 linux art research platform.

Prerequisites

Assume Fedora 27.

Background

The wide variety of web and print platforms for contemporary artistic research can be found at the survey.

Components

  1. Database
  2. Webserver
  3. Content Management System
  4. Wiki
  5. Etherpad
  6. IPFS

TensorFlow Configuration and Optimization Notes

Notes for installing TensorFlow on linux, with GPU enabled.

Background

TensorFlow is the second-generation ML framework from Google. (See this comparison of deep learning software.) The current state-of-the art image recognition models (inception-v3) use this framework.

Prequisites

Assuming Fedora 24 with Nvidia 1060 installed, running nvidia as opposed to nouveau drivers. See Fedora 24 Notes, and RPM Fusion’s installation page for installing the Nvidia drivers. In sum,

dnf install -y xorg-x11-drv-nvidia akmod-nvidia "kernel-devel-uname-r == $(uname -r)"
dnf install xorg-x11-drv-nvidia-cuda
dnf install vulkan

After, install some devel packages.

dnf install -y vulkan-devel

Download the Nvidia GPU CUDA Toolkit. The version used for this install is 8.0.61, and the network install for Fedora x86_64 was used.

This version of CUDA Toolkit is not C++11/C++14/C++17 aware. So, be aware! One way around this is to mod like below, and use -std=gnu++98.

117c117,118
 5
---
> /* bkoz use -std=c++98 if necessary */
> #if __GNUC__ > 6

Next, compile top-of-tree OpenCV (aka 3.2) with CUDA enabled. To do so, use the following configure list, mod for paths on system:

cmake -DVERBOSE=1 -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -std=gnu++98 -Wno-deprecated-gpu-targets" -D BUILD_EXAMPLES=1 -D BUILD_DOCS=1 -D WITH_OPENNI=1 -D WITH_CUDA=1 -D CUDA_FAST_MATH=1 -D WITH_CUBLAS=1 -D WITH_FFMPEG=1 -D WITH_EIGEN=1 -D ENABLE_FAST_MATH=1 -D ENABLE_SSE3=1 -D ENABLE_AVX=1 -D CMAKE_BUILD_TYPE=RELEASE -D ENABLE_PRECOMPILED_HEADERS=OFF  -D CMAKE_INSTALL_PREFIX=/home/bkoz/bin/H-opencv -D OPENCV_EXTRA_MODULES_PATH=/home/bkoz/src/opencv_contrib.git/modules /home/bkoz/src/opencv.git/

Admittedly, this abuse of CMAKE_CXX_FLAGS is not optimal. Maybe EXTRA_CXX_FLAGS?

Now, for Nvidia cuDNN. The version used for this install is 5.1

When that is done, use pip to install TensorFlow.

sudo pip install --upgrade pip;
sudo pip install tensorflow-gpu

This should output something like:

Collecting tensorflow-gpu
  Downloading tensorflow_gpu-0.12.1-cp27-cp27mu-manylinux1_x86_64.whl (89.7MB)
    100% |████████████████████████████████| 89.7MB 19kB/s 
Requirement already satisfied: mock>=2.0.0 in /usr/lib/python2.7/site-packages (from tensorflow-gpu)
Requirement already satisfied: six>=1.10.0 in /usr/lib/python2.7/site-packages (from tensorflow-gpu)
Requirement already satisfied: numpy>=1.11.0 in /usr/lib64/python2.7/site-packages (from tensorflow-gpu)
Collecting protobuf>=3.1.0 (from tensorflow-gpu)
  Downloading protobuf-3.2.0-cp27-cp27mu-manylinux1_x86_64.whl (5.6MB)
    100% |████████████████████████████████| 5.6MB 284kB/s 
Collecting wheel (from tensorflow-gpu)
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 3.3MB/s 
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow-gpu)
Requirement already satisfied: funcsigs>=1 in /usr/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow-gpu)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from protobuf>=3.1.0->tensorflow-gpu)
Installing collected packages: protobuf, wheel, tensorflow-gpu
Successfully installed protobuf-3.2.0 tensorflow-gpu-0.12.1 wheel-0.29.0

After this has completed, add in Keras.

Optimization

For Nvidia GPUs, take a look at this interesting post from Netflix. In sum, add

NVreg_CheckPCIConfigSpace=0

Notes on the Deep Deep Deepest

 

Reading

 

Approaches

  • SVM (Support Vector Machines)
  • RBM (Restricted Boltzmann Machines)
  • NN/Convolution NN/DNN

 

Silicon Valley Fun

TensorFlow Dev Summit

February 15, 2017 @ googleplex, Mountain View, CA

 

Software

theano

  • python-theano-doc
  • python3-theano
  • python2-theano

tensorflow

  • TensorFlow
  • github
  • Current models for facial recognition include VGG-19, VGG-16, and inception-v3. Of the listed models, inception-v3 seems to have the advantage, at least as of early 2017.

keras

scikit-learn

opencv

 

GPU Hardware

Recommended GPUS are: Nvidia GTX 1080, 1070, 980, and 970. Maximize CUDA cores.

Fedora 25 Notes

Install:
San Francisco, California USA

Using x86_64 netinstall images as install media.

Update/Post-Install:

Use RPM Fusion Repositories.

Some basics

dnf install -y tcsh gnome-tweak-tool livecd-tools ncftp lftp

Install KDE, Razor-qt, and MATE Desktops

	 	 
dnf install -y lightdm-gtk-greeter-settings mate-desktop mate-applets nm-connection-editor network-manager-applet caja mate-themes bluecurve-gtk-themes gnome-themes-legacy;
dnf groupinstall mate-desktop;  
dnf groupinstall kde-desktop;
dnf install -y gnome-themes;

Optionally mate-compiz can be installed:

dnf groupinstall mate-compiz;

Install the basics for media and VLC

dnf install -y vlc vlc-core vlc-extras;
dnf install -y gaupol mediainfo

Install media xtra codecs.

dnf install -y ffmpeg ffmpeg-devel ffmpeg-compat-devel gstreamer-plugins-bad gstreamer-plugins-ugly gstreamer-plugins-good gstreamer-plugins-good-extras gstreamer-plugins-bad-free gstreamer-plugins-bad-free-extras gstreamer-plugins-bad-nonfree gstreamer-ffmpeg;
dnf install -y k3b-extras-freeworld;

Install negativo.
Use negativo17 repositories for HandBrake, volia.

dnf config-manager --add-repo=http://negativo17.org/repos/fedora-handbrake.repo;
dnf install -y HandBrake-gui HandBrake-cli;
dnf install -y makemkv mkvtoolnix mkvtoolnix-gui;
dnf install -y libdvdcss;

Use makemkv for Bluray.

Install some editors, utilities, and basics

dnf install -y calligra-words abiword emacs

Install devel 1

dnf install -y openni openni-devel openni-doc  tbb tbb-devel tbb-doc eigen3-devel eigen3-doc openrdate simple-mtpfs emacs python-BeautifulSoup

Install devel 2

dnf groupinstall -y "Development Tools";
dnf install -y gcc-c++ libstdc++-devel libstdc++-docs boost boost-devel boost-doc valgrind gdb elfutils nemiver dwarves dejagnu emacs expect flex bison cmake git-svn mpfr-devel libmpc-devel lsb;
dnf debuginfo-install glibc;

Install database

dnf install -y glom libpqxx-devel libpqxx postgresql-devel

Install computer vision (hand-build clandmarks and opencv if optimized)

dnf install -y CImg-devel rapidxml-devel opencv opencv-devel

Install graphic

dnf install -y freetype-freeworld inkscape* ufraw ufraw-gimp fuse-exfat

Install technical documentation toolchain

dnf install -y pdfedit texlive-pdfjam doxygen doxygen-latex graphviz  docbook5-style-xsl docbook5-schemas docbook2X docbook-style-xsl dblatex texinfo-tex dblatex docbook-utils-pdf  graphviz-python graphviz-R graphviz-graphs graphviz-doc R-core

Install sysadmin

sudo dnf install -y system-config-network system-config-samba system-config-services system-config-users system-config-firewall system-config-httpd

See separate posts on getting Nvidia and TensorFlow optimized CUDA up.

For rendering svg, use both inkscape and cairosvg. To install the latter, do

pip3 install cairosvg

Issues

Something wrong with file indexing, it is consuming gigabytes of memory. Turn off search in the Gnome menu, but this is insufficient. Instead, use

balooctl disable

Set ulimits higher. Modify /etc/security/limits.cof like so:

*** limits.conf.f25	2016-12-11 23:27:46.609458649 -0800
--- limits.conf	2016-12-15 20:16:57.289245497 -0800
***************
*** 52,57 ****
--- 52,58 ----
  
  #*               soft    core            0
  #*               hard    rss             10000
+ 
  #@student        hard    nproc           20
  #@faculty        soft    nproc           20
  #@faculty        hard    nproc           50
***************
*** 59,61 ****
--- 60,67 ----
  #@student        -       maxlogins       4
  
  # End of file
+ 
+ bkoz               hard    nofile	8092
+ bkoz               soft    nofile       8092
+ 
+ 

In addition, it seems as if there is a bug with nmcli when using the Mate window manager. Mostly, it doesn’t work (along with nm-applet when used in a programmatic fashion from C/C++ code. So, use GNOME, as much as it hurts.

Fedora 24 Notes

Install:
San Francisco, California USA

Using x86_64 netinstall images as install media.

Update/Post-Install:

Use RPM Fusion Repositories.

Some basics

dnf install -y unrar tcsh gnome-tweak-tool livecd-tools ncftp lftp

Install KDE, Razor-qt, and MATE Desktops

	 	 
dnf install -y lightdm-gtk-greeter-settings mate-desktop mate-applets network-manager-applet caja mate-themes bluecurve-gtk-themes gnome-themes-legacy;
dnf groupinstall mate-desktop;  
dnf groupinstall kde-desktop;
dnf install -y gnome-themes;

Still a little sketchy, but optionally mate-compiz can be installed:

dnf groupinstall mate-compiz;

Install the basics for VLC

dnf install -y vlc vlc-core vlc-extras;
dnf install -y gaupol mediainfo

Install media xtra codecs.

dnf install -y ffmpeg ffmpeg-devel ffmpeg-compat-devel gstreamer-plugins-bad gstreamer-plugins-ugly gstreamer-plugins-good gstreamer-plugins-good-extras gstreamer-plugins-bad-free gstreamer-plugins-bad-free-extras gstreamer-plugins-bad-nonfree gstreamer-ffmpeg;
dnf install -y k3b-extras-freeworld;

Install negativo.
Use negativo17 repositories for HandBrake, volia.

dnf config-manager --add-repo=http://negativo17.org/repos/fedora-handbrake.repo;
dnf install -y HandBrake-gui HandBrake-cli;
dnf install -y makemkv mkvtoolnix mkvtoolnix-gui;
dnf install -y libdvdcss;

Use makemkv for Bluray.

Install some editors, utilities, and basics

dnf install -y abiword emacs tcsh

Install devel 1

dnf install -y openni openni-devel openni-doc  tbb tbb-devel tbb-doc eigen3-devel eigen3-doc openrdate simple-mtpfs emacs python-BeautifulSoup

Install devel 2

dnf groupinstall -y "Development Tools";
dnf install -y gcc-c++ libstdc++-devel libstdc++-docs boost boost-devel boost-doc valgrind gdb elfutils nemiver dwarves dejagnu emacs expect flex bison cmake git-svn mpfr-devel libmpc-devel lsb;
dnf debuginfo-install glibc;

Install database

dnf install -y glom libpqxx-devel libpqxx postgresql-devel

Install computer vision (hand-build clandmarks)

dnf install -y CImg-devel rapidxml-devel opencv opencv-devel

Install graphic

dnf install -y freetype-freeworld inkscape* ufraw ufraw-gimp 

Install technical documentation toolchain

dnf install -y pdfedit doxygen doxygen-latex graphviz  docbook5-style-xsl docbook5-schemas docbook2X docbook-style-xsl dblatex texinfo-tex dblatex docbook-utils-pdf  graphviz-python graphviz-R graphviz-graphs graphviz-doc R-core

Install sysadmin

sudo dnf install -y system-config-network system-config-samba system-config-services system-config-users system-config-firewall system-config-httpd