Internet Life, Social Media, Backup and Archiving

Facebook

There is a plugin called “Give Me My Data” which, when installed, can be used to export the wall data from Facebook, along with friend and mutual friend graph information in DOT and XML formats.

Google, g+, gmail, youtube.

One intriguing option is google takeout, which will package up all your metadata into one (or more) files.  In addition, gmail can be archived via POP to local mailclient and then exported to mbox or other. Chrome bookmarks can be exported, Google maps can be exported, etc. Google plus data can be shaped by user-configurable settings.

Data liberation front, nice job!

Tumblr

This one is a bit complicated.There is no tumblr export or backup capability provided by tumblr.com, so instead a variety of other options exist. The most reliable is to import your tumblr blog into WordPress, and then export it from WordPress in an XML form.

To do this, see the tumblr.com post.

After import has completed, all the tumbler posts will be imported into the target wordpress.com blog as individual posts. Some of these posts will have titles, some will have the tumbler post id number as the title. All posts are imported with with visibility set to “Public.” Tags in tumblr posts are converted to wordpress tags, and all tumblr posts will have the initial category of “Uncategorized.”

Apply the following transformations to the imported tumblr posts:

1. Create a new “tumblr” category and tag, and batch process the imported posts categorized as “Uncategorized” with it.

2. Set visibility to “Private” 

3. Turn off comments, pings, etc.

These are all easier if the default 20 posts per page is set to a larger number, ie to 999. To do this, Enter the “Posts” menu, and select the top right “Screen Options” button. Change 20 posts to 100. Technically, one can go up to 999 posts per page, but then the query generated by the large bulk transformations makes hosted wordpress/client browsers fail.YMMV. The 100 to 200 range seems to work well for my situation.

To help organize the imported tumblr posts, I create tags for “tumblr”, “import,” and the name of the tumblr blog imported. A tumblr category is made. Then, these are bulk applied and the posts marked “Private” and categories set to “tumblr.”

The tumblr import is remarkably intact. Tags, posts, titles, etc. Photos, videos are imported (although multi-photo arrangements in tumblr may be imported with different orientation.) I’m starting to like the wordpress archive better than the new tumblr dashboard, to be quite honest.

Adding to the oddity that is import/export from tumblr, see wp2tumblr on github. This was the opposite idea: take wordpress content and then import it into tumblr.

Twitter

Here is a post from twitter about your twitter archive.

WordPress

Sign in to the Dashboard, and go to the “Tools” option on the left menu. Select export. Volia. Easy peasy.

Static Linking for C++ Shared Objects

Prerequisites

Assuming recent-vintage linux with standard C/C++ developer setup, including g++, readelf, ld, gold etc., etc. Reference platform is Fedora 17 (F17) using gcc-4.7.2. Linking static libraries may require the installation of non-default packages. For C/C++, this means:

yum install -y glibc-static libstdc++-static

Problem Description

Create a shared library that uses C++ internally, but with an external C interface and all dependent libraries statically-linked.

Some quick background/primer on creating a shared library with current GNU tools on recent linux. Consider the default case, making a shared library.

Take the first source file, a generic library function implemented in C++ and exported via extern C.

// filename: 28811.cc
#include <iostream>
#include <string>

extern "C" void announce()
{
  const std::string s("oooohllalala");
  std::cout << s << std::endl;
}

And then the second file, which uses it.

// filename: 28811-loop.cc
extern "C" void announce();

int main()
{
  announce();
  return 0;
}

Compile via:

g++ -shared -fPIC -O2 -g 28811.cc -o libannounce.so

g++ -g -O2 -L. 28811-loop.cc -lannounce -o 28811.exe

To run the executable, first make sure that LD_RUN_PATH has the directory containing the shared library libannounce.so. Then:

28811.exe

Which should print:

oooohllalala

Now that that is confirmed to be working, here’s a closer look at the library file libannounce.so that was produced.

A look at the linked dependencies:

%ldd libannounce.so 
	linux-vdso.so.1 =>  (0x00007fffed1ff000)
	libstdc++.so.6 => /mnt/share/bld/gcc.git-trunk/x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6 (0x00007fd87e7e5000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fd87e4c3000)
	libgcc_s.so.1 => /mnt/share/bld/gcc.git-trunk/gcc/libgcc_s.so.1 (0x00007fd87e4ad000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fd87e0f6000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003b72200000)

This shows that the support library has a depedent link on the C language runtime (ie libc.so and libm.so, the GNU C/C++ language support libray (libgcc_s.so), the C++ language runtime (libstdc++.so), and the linux dynamic loader (linux-vdso.so).

A look at what’s in the library:

readelf -s libannounce.so | grep announce
    25: 0000000000000d20   226 FUNC    GLOBAL DEFAULT   11 announce
    61: 0000000000000d20   226 FUNC    GLOBAL DEFAULT   11 announce

This is our baseline setup. Now, try some variations.

The first variation: try with libgcc_s.so statically-linked.

g++ -shared -static-libgcc -fPIC -O2 -g 28811.cc -o libannounce.so

And then look at the library produced:

%ldd libannounce.so 
	linux-vdso.so.1 =>  (0x00007fff075ff000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f27ceeb9000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f27cebbd000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f27ce806000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003b72200000)
	libgcc_s.so.1 => /mnt/share/bld/gcc.git-trunk/gcc/libgcc_s.so.1 (0x00007f27ce7f0000)

It’s missing libgcc_s.so, as expected.

The second variation: try with libgcc_s.so and libstdc++.so statically-linked.

So:

g++ -shared -static-libgcc -static-libstdc++ -fPIC -O2 -g 28811.cc -o libannounce.so

Gives:

%ldd libannounce.so 
	linux-vdso.so.1 =>  (0x00007fff819ff000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fdba493a000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fdba4582000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003b72200000)

As expected.

Let’s try a third variation, to try for an all-C static link too. Ie:

g++ -shared -static-libgcc -static-libstdc++ -fPIC -O2 -g 28811.cc -Wl,-Bstatic -lc -lm -o libannounce.so

This doesn’t work:

g++ -shared -static-libgcc -static-libstdc++ -fPIC -O2 -g 28811.cc -Wl, -static -lc -lm -o libannounce.so
/usr/bin/ld: /mnt/share/bin/H-x86_64-gcc/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.0/crtbeginT.o: relocation R_X86_64_32 against `__TMC_END__' can not be used when making a shared object; recompile with -fPIC
/mnt/share/bin/H-x86_64-gcc/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.0/crtbeginT.o: could not read symbols: Bad value
collect2: error: ld returned 1 exit status


Known Bugs

PR28811
PR54482

Older bugs:

PR52689

Some of these bugs are many years old, and the build machinery has changed since they were first filed.

Basics on impacted sources:

src/c++98/compatibility.cc
src/c++98/compatibility-list-2.cc
src/c++11/compatibility-c++0x.cc
src/c++11/compatibility-atomic-c++0x.cc
src/c++11/compatibility-thread-c++0x.cc

and configure via PIC_CXXFLAGS

src/c++98/Makefile.am
src/c++11/Makefile.am

testing via

./lib/libstdc++.exp
     [list "incdir=$srcdir" "additional_flags=-w -shared 
testsuite/17_intro/static.cc:
     { dg-options "-static-libstdc++ -std=gnu++11" }
testsuite/17_intro/static_pic.cc:
     { dg-options "-shared -fPIC -static-libgcc -static-libstdc++"

It looks like the C++ compiler testsuite has a bunch of -fPIC tests, some -fPIC -fvisibility=hidden tests (mostly in the context of local statics), and some -static tests.

Questions

1) Are these expectations valid for C too?

First, what the GCC Manual says about the specific link options. Use “N1975 Dynamic Shared Objects: Survey and Issues” as background.

Then figure out the uses for:

  • -static
  • partial static shorcuts: -static-libgccgcc, -static-libstdc++
  • partial static via:
    -Wl,-static -lssl -lcrypto

2) how to check to see if an object has relocations

readelf -r foo.o

Ian Taylor has a nice description of relocations.