Hello all,

I wanted to build TensorFlow from sources on a computer that doesn’t have access to the Internet. After some research on the Internet I didn’t find anyone successfully managing that specifically with CUDA support. There is an offline helper tensorflow-offline (github.com) by @amutu but that supports TensorFlow version 1.2.1. To be fair I didn’t try if it would work for the currently latest one (1.5.0), but I need CUDA support.

TL;DR Configure application, bazel fetch and bazel build for a few seconds on an online computer using “bazel –output_user_root=`pwd`/../tf_tmp %REST_OF_COMMAND%. Package and transfer ../tf_tmp/%HASH1%/external to the remote. Identically configure application, build same as the online (it will fail), extract package to ../tf_tmp/%HASH2%/external and then build again. Voila!

The suggested solution is universal, for other bazel based applications as well.

Here it goes!

This will require to create twice the developer environment. For me this means more or less having cuda (9.1 locally and 9.0 on the server) installed, cudnn (7.0 both) and nccl (2.1.4-1), using bazel release 0.10.1.

Setup both environments identically, as seen on the build guide (tensorflow.org).

Online server

Run (check * Important in the end of the post):

 > bazel --output_user_root=`pwd`/../tf_tmp fetch --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Until it fully succeeds, you might need to run it multiple times or delete ../tf_tmp/install folder to succeed. Then start building the application because there might be some packages that will be checked at that moment:

 > bazel --output_user_root=​pwd`/../tf_tmp build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

exit after a few seconds, when you see it has started to compile.

Navigate to ../tf_tmp/%HASH1%/ where 22b7191bf872641ec533c3a935b4af91 is my hash, i.e.

 > cd ../tf_tmp/22b7191bf872641ec533c3a935b4af91/

Create an archive with the external folder

 > tar czf external.tar.gz external

Offline server:

Execute:

 > bazel --output_user_root=​pwd`/../tf_tmp build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

This will create the folder structure in tf_tmp so we can add the external folder

Online server:

Move to the offline server:

 > scp external.tar.gz server:/path/on/remote

Offline server:

extract external.tar.gz to /path/to/tf_tmp/%HASH2%/

 > tar xzf external.tar.gz -C $HOME/tf_tmp/7d826155cbbfce6e938d5e7e034e981b/

Build TensorFlow and it will succeed.

If by any chance you change the location of the TensorFlow source path or rename the folder,  you will need to extract external.tar.gz again.

* Important: this only adds the –output_user_root parameter which should not be withing the same folder as the source code. When the bazel command is

 > bazel build

this becomes

 > bazel --output_user_root=​/some/other/path build

solution based on @truatpasteurdotfr’s (github.com) idea.

Hope this helps someone

— flanaras

— update [2018/03/16]: Add credits and bazel version

Advertisements