IPFS Background

Background

Some info:
Stanford Seminar, IPFS and the Permanent Web, October 22, 2015.

Main problem points with current web: offline, distributed, permanence, security, speed, lack of richer communication protocols.

Merkle Web, Merkle Tree, Merkle Links: protocol to upgrade how the web works. Related to git’s merkledag. Instead of using a centralized address to locate, use a combination of data x hash function as the address, that way it can be shared.

A quick summary:

Taking a higher-level approach to network design, like Paul Baran and some of the earliest categorization of networks, as featured in 1964’s On Distributed Communications. See network designs for centralized, distributed, peer, isolated, etc.

There is also no ‘presence linking’ in the status quo, meaning that there isn’t a notion for a peer to announce itself in several transports, so that other peers can guarantee that it is always the same peer.

This is especially problematic when sharing media. Any file, any media: when shared on a centralized network, the total amount of data required to share is the size of the original media times multiplied by the number of shares (modulo caching, if be). Ten or 20x multiples are common.

The solution is a distributed web with content uniquely identified, so it can be properly served by multiple peers in a resilient manner. In addition, this works for disconnected, offline, and slow networks.

Instead, social platforms are throwing tons of capital to build walled gardens on centralized networks. Apps continue to dominate the mobile web.

Some hype:
TEDxSanFrancisco, The Next Internet Revolution, December, 2016.

There is documentation for the distributed web as a free gitbook, The Decentralized Web Primer.

Sources, Protocol Specifications

InterPlanetary File System wikipedia.

The IPFS Project, a hypermedia transport protocol project. Based on a network stack encoded in libp2p. See specs documentation here. The naming system is IPNS.

Online community is found here.

Part of Protocol Labs

Questions
what about other peer to peer networking libraries, namely libtorrent? libp2p design compare?

language bindings for libp2p are javascript, go, rust, python. What else is in development?

See Also

Dtube

Artistic Research Platform Notes

Software configuration for commodity x86/86_64 linux art research platform.

Prerequisites

Assume Fedora 27.

Background

The wide variety of web and print platforms for contemporary artistic research can be found at the survey.

Components

  1. Database
  2. Webserver
  3. Content Management System
  4. Wiki
  5. Etherpad
  6. IPFS

Dell XPS 13 9360 Notes

Linux needs certain BIOS settings flipped: SATA Operation to AHCI, Secure Boot Disabled

TensorFlow Configuration and Optimization Notes

Notes for installing TensorFlow on linux, with GPU enabled.

Background

TensorFlow is the second-generation ML framework from Google. (See this comparison of deep learning software.) The current state-of-the art image recognition models (inception-v3) use this framework.

Prequisites

Assuming Fedora 24 with Nvidia 1060 installed, running nvidia as opposed to nouveau drivers. See Fedora 24 Notes, and RPM Fusion’s installation page for installing the Nvidia drivers. In sum,

dnf install -y xorg-x11-drv-nvidia akmod-nvidia "kernel-devel-uname-r == $(uname -r)"
dnf install xorg-x11-drv-nvidia-cuda
dnf install vulkan

After, install some devel packages.

dnf install -y vulkan-devel

Download the Nvidia GPU CUDA Toolkit. The version used for this install is 8.0.61, and the network install for Fedora x86_64 was used.

This version of CUDA Toolkit is not C++11/C++14/C++17 aware. So, be aware! One way around this is to mod like below, and use -std=gnu++98.

117c117,118
 5
---
> /* bkoz use -std=c++98 if necessary */
> #if __GNUC__ > 6

Next, compile top-of-tree OpenCV (aka 3.2) with CUDA enabled. To do so, use the following configure list, mod for paths on system:

cmake -DVERBOSE=1 -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -std=gnu++98 -Wno-deprecated-gpu-targets" -D BUILD_EXAMPLES=1 -D BUILD_DOCS=1 -D WITH_OPENNI=1 -D WITH_CUDA=1 -D CUDA_FAST_MATH=1 -D WITH_CUBLAS=1 -D WITH_FFMPEG=1 -D WITH_EIGEN=1 -D ENABLE_FAST_MATH=1 -D ENABLE_SSE3=1 -D ENABLE_AVX=1 -D CMAKE_BUILD_TYPE=RELEASE -D ENABLE_PRECOMPILED_HEADERS=OFF  -D CMAKE_INSTALL_PREFIX=/home/bkoz/bin/H-opencv -D OPENCV_EXTRA_MODULES_PATH=/home/bkoz/src/opencv_contrib.git/modules /home/bkoz/src/opencv.git/

Admittedly, this abuse of CMAKE_CXX_FLAGS is not optimal. Maybe EXTRA_CXX_FLAGS?

Now, for Nvidia cuDNN. The version used for this install is 5.1

When that is done, use pip to install TensorFlow.

sudo pip install --upgrade pip;
sudo pip install tensorflow-gpu

This should output something like:

Collecting tensorflow-gpu
  Downloading tensorflow_gpu-0.12.1-cp27-cp27mu-manylinux1_x86_64.whl (89.7MB)
    100% |████████████████████████████████| 89.7MB 19kB/s 
Requirement already satisfied: mock>=2.0.0 in /usr/lib/python2.7/site-packages (from tensorflow-gpu)
Requirement already satisfied: six>=1.10.0 in /usr/lib/python2.7/site-packages (from tensorflow-gpu)
Requirement already satisfied: numpy>=1.11.0 in /usr/lib64/python2.7/site-packages (from tensorflow-gpu)
Collecting protobuf>=3.1.0 (from tensorflow-gpu)
  Downloading protobuf-3.2.0-cp27-cp27mu-manylinux1_x86_64.whl (5.6MB)
    100% |████████████████████████████████| 5.6MB 284kB/s 
Collecting wheel (from tensorflow-gpu)
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 3.3MB/s 
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow-gpu)
Requirement already satisfied: funcsigs>=1 in /usr/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow-gpu)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from protobuf>=3.1.0->tensorflow-gpu)
Installing collected packages: protobuf, wheel, tensorflow-gpu
Successfully installed protobuf-3.2.0 tensorflow-gpu-0.12.1 wheel-0.29.0

After this has completed, add in Keras.

Optimization

For Nvidia GPUs, take a look at this interesting post from Netflix. In sum, add

NVreg_CheckPCIConfigSpace=0

Fedora 25 Nvidia Notes

Step one: install Fedora 25.

Step two: turn off Wayland as the default. This is pretty simple, ie change /etc/gdm/custom.conf

WaylandEnable=false

When the GUI is started again, the following (assuming the current session is 2)

loginctl show-session 2 -p Type

Shows:

Type=x11

Then look at the RPM Fusion page for Nvidia.

Step three: Nvidia drivers are not working on Fedora 25. Instead, revert to Fedora 24 and proceed as above.

Notes on the Deep Deep Deepest

 

Reading

 

Approaches

  • SVM (Support Vector Machines)
  • RBM (Restricted Boltzmann Machines)
  • NN/Convolution NN/DNN

 

Silicon Valley Fun

TensorFlow Dev Summit

February 15, 2017 @ googleplex, Mountain View, CA

 

Software

theano

  • python-theano-doc
  • python3-theano
  • python2-theano

tensorflow

  • TensorFlow
  • github
  • Current models for facial recognition include VGG-19, VGG-16, and inception-v3. Of the listed models, inception-v3 seems to have the advantage, at least as of early 2017.

keras

scikit-learn

opencv

 

GPU Hardware

Recommended GPUS are: Nvidia GTX 1080, 1070, 980, and 970. Maximize CUDA cores.

Cartography Futures

1.dymaxion to authagraph to myriahedral projections
Buckminister Fuller and co-cartographer Shoji Sadao, designed an alternative projection map, called the Dymaxion map. Since then, several other projections have been proposed that evolve the idea of map projections in a similar manner. Favorites are:
2. new approaches to population density, use natural earth data and then read the shapefile with python

 

4. geospatial python
http://geospatialpython.com/

 

5. example api