1###################################### 2Dynamic Linking and Loading with glibc 3###################################### 4 5.. contents:: 6 :local: 7 :backlinks: none 8 :depth: 2 9 10.. Note:: 11 :class: caution 12 13 Portable Native Client currently only supports static linking, and the 14 only C library available for it is newlib. This page is only valid for 15 Native Client, though PNaCl will eventually support some form of 16 dynamic linking. 17 18This document describes how to create and deploy dynamically linked and loaded 19applications with the glibc library in the Native Client SDK. Before reading 20this document, we recommend reading :doc:`Building Native Client Modules 21<building>` 22 23.. _c_libraries: 24 25C standard libraries: glibc and newlib 26-------------------------------------- 27 28The Native Client SDK comes with two C standard libraries --- glibc and 29newlib. These libraries are described in the table below. 30 31+-----------------------------------------------------+----------+-------------+ 32| Library | Linking | License | 33+=====================================================+==========+=============+ 34|glibc | dynamic | GNU Lesser | 35| The GNU implementation of the POSIX_ standard | or static| General | 36| runtime library for the C programming language. | | Public | 37| Designed for portability and performance, glibc is | | License | 38| one of the most popular implementations of the C | | (LGPL) | 39| library. It is comprised of a set of interdependent| | | 40| libraries including libc, libpthreads, libdl, and | | | 41| others. For documentation, FAQs, and additional | | | 42| information about glibc, see GLIBC_. | | | 43+-----------------------------------------------------+----------+-------------+ 44|newlib | static | Berkeley | 45| newlib is a C library intended for use in embedded | | Software | 46| systems. Like glibc, newlib is a conglomeration of | | Distribution| 47| several libraries. It is available for use under | | (BSD) type | 48| BSD-type free software licenses, which generally | | free | 49| makes it more suitable to link statically in | | software | 50| commercial, closed-source applications. For | | licenses | 51| documentation, FAQs, and additional information | | | 52| about newlib, see newlib_. | | | 53+-----------------------------------------------------+----------+-------------+ 54 55 56For proprietary (closed-source) applications, your options are to either 57statically link to newlib, or dynamically link to glibc. We recommend 58dynamically linking to glibc, for a couple of reasons: 59 60* The glibc library is widely distributed (it's included in Linux 61 distributions), and as such it's mature, hardened, and feature-rich. Your 62 code is more likely to compile out-of-the-box with glibc. 63 64* Dynamic loading can provide a big performance benefit for your application if 65 you can structure the application to defer loading of code that's not needed 66 for initial interaction with the user. It takes some work to put such code in 67 shared libraries and to load the libraries at runtime, but the payoff is 68 usually worth it. In future releases, Chrome may also support caching of 69 common dynamically linked libraries such as libc.so between applications. 70 This could significantly reduce download size and provide a further potential 71 performance benefit (for example, the hello_world example would only require 72 downloading a .nexe file that's on the order of 30KB, rather than a .nexe 73 file and several libraries, which are on the order of 1.5MB). 74 75Native Client support for dynamic linking and loading is based on glibc. Thus, 76**if your Native Client application must dynamically link and load code (e.g., 77due to licensing considerations), we recommend that you use the glibc 78library.** 79 80.. Note:: 81 :class: note 82 83 **Disclaimer:** 84 85 * **None of the above constitutes legal advice, or a description of the legal 86 obligations you need to fulfill in order to be compliant with the LGPL or 87 newlib licenses. The above description is only a technical explanation of 88 the differences between newlib and glibc, and the choice you must make 89 between the two libraries.** 90 91 92 93.. Note:: 94 :class: note 95 96 **Notes:** 97 98 * Static linking with glibc is rarely used. Use this feature with caution. 99 100 * The standard C++ runtime in Native Client is provided by libstdc++; this 101 library is independent from and layered on top of glibc. Because of 102 licensing restrictions, libstdc++ must be statically linked for commercial 103 uses, even if the rest of an application is dynamically linked. 104 105SDK toolchains 106-------------- 107 108The Native Client SDK contains multiple toolchains, which are differentiated by 109:ref:`target architecture <target_architectures>` and C library: 110 111=================== ========= =============================== 112Target architecture C library Toolchain directory 113=================== ========= =============================== 114x86 newlib toolchain/<platform>_x86_newlib 115x86 glibc toolchain/<platform>_x86_glibc 116ARM newlib toolchain/<platform>_arm_newlib 117PNaCl newlib toolchain/<platform>_pnacl 118=================== ========= =============================== 119 120In the directories listed above, <platform> is the platform of your development 121machine (i.e., win, mac, or linux). For example, in the Windows SDK, the x86 122toolchain that uses glibc is in ``toolchain/win_x86_glibc``. 123 124.. Note:: 125 :class: note 126 127 **Note:** The ARM and PNaCl toolchains are currently restricted to newlib. 128 129To use the glibc library and dynamic linking in your application, you **must** 130use a glibc toolchain. (Currently the only glibc toolchain is 131``<platform>_x86_glibc``.) Note that you must build all code in your application 132with one toolchain. Code from multiple toolchains cannot be mixed. 133 134Specifying and delivering shared libraries 135------------------------------------------ 136 137One significant difference between newlib and glibc applications is that glibc 138applications must explicitly list and deploy the shared libraries that they 139use. 140 141In a desktop environment, when the user launches a dynamically linked 142application, the operating system's program loader determines the set of 143libraries the application requires by reading explicit inter-module 144dependencies from executable file headers, and loads the required libraries 145into the address space of the application process. Typically the required 146libraries will have been installed on the system as a part of the application's 147installation process. Often the desktop application developer doesn't know or 148think about the libraries that are required by an application, as those details 149are taken care of by the user's operating system. 150 151In the Native Client sandbox, dynamic linking can't rely in the same way on the 152operating system or the local file system. Instead, the application developer 153must identify the set of libraries that are required by an application, list 154those libraries in a Native Client :ref:`manifest file <manifest_file>`, and 155deploy the libraries along with the application. Instructions for how to build 156a dynamically linked Native Client application, generate a Native Client 157manifest (.nmf) file, and deploy an application are provided below. 158 159Building a dynamically linked application 160========================================= 161 162Applications built with the glibc toolchain will by dynamically linked by 163default. Application that load shared libraries at runtime using ``dlopen()`` 164must link with the libdl library (``-ldl``). 165 166Like other gcc-based toolchains building a dynamic library for NaCl is normally 167done by linking with the ``-shared`` flag and compiling with the ``-fPIC`` flag. 168The SDK build system will do this automatically when the ``SO_RULE`` Makefile 169rule is used. 170 171The Native Client SDK includes an example that demonstrates how to build a 172shared library, and how to use the ``dlopen()`` interface to load that library 173at runtime (after the application is already running). Many applications load 174and link shared libraries at launch rather than at runtime, and hence do not 175use the ``dlopen()`` interface. The SDK example is nevertheless instructive, as 176it demonstrates how to build Native Client modules (.nexe files) and shared 177libraries (.so files) with the x86 glibc toolchain, and how to generate a 178Native Client manifest file for glibc applications. 179 180The SDK example, located in ``examples/tutorial/dlopen``, includes three C++ 181files: 182 183eightball.cc 184 This file implements the function ``Magic8Ball()``, which is used to provide 185 whimsical answers to user questions. This file is compiled into a shared 186 library called ``libeightball.so``. This library gets included in the 187 .nmf file and is therefore directly loadable with ``dlopen()``. 188 189reverse.cc 190 This file implements the function ``Reverse()``, which returns reversed 191 copies of strings that are passed to it. This file is compiled into a shared 192 library called ``libreverse.so``. This library is **not** included in the 193 .nmf file and is loaded via an http mount using the :ref:`nacl_io library 194 <nacl_io>`. 195 196dlopen.cc 197 This file implements the Native Client module, which loads the two shared 198 libraries and handles communcation with with JavaScript. The file is compiled 199 into a Native Client executable (.nexe). 200 201Run ``make`` in the dlopen directory to see the commands the Makefile executes 202to build x86 32-bit and 64-bit .nexe and .so files, and to generate a .nmf 203file. These commands are described below. 204 205.. Note:: 206 :class: note 207 208 **Note:** The Makefiles for most of the examples in the SDK build the 209 examples using multiple toolchains (x86 newlib, x86 glibc, ARM, and PNaCl). 210 With a few exceptions (listed in the :ref:`Release Notes 211 <sdk-release-notes>`), running "make" in each example's directory builds 212 multiple versions of the example using the SDK toolchains. The dlopen example 213 is one of those exceptions – it is only built with the x86 glibc toolchain, 214 as that is currently the only toolchain that supports glibc and thus dynamic 215 linking and loading. Take a look at the example Makefiles and the generated 216 .nmf files for details on how to build dynamically linked applications. 217 218.. _dynamic_loading_manifest: 219 220Generating a Native Client manifest file for a dynamically linked application 221============================================================================= 222 223The Native Client manifest file specifies the name of the executable to run 224and must also specify any shared libraries that the application directly 225depends on. For indirect dependencies (such as libraries opened via 226``dlopen()``) it is also convenient to list libraries in the manifest file. 227However it is possile to load arbitrary shared libraries at runtime that 228are not mentioned in the manifest by using the `nacl_io library <nacl_io>`_ 229to mount a filesystem that contains the shared libraries which will then 230allow ``dlopen()`` to access them. 231 232In this example we demonstrate both loading directly from via the manifest 233file (``libeightball.so``) and loading indirectly via a http mount 234(``libreverse.so``). 235 236Take a look at the manifest file in the dlopen example to see how 237a glibc-style manifest file is structured. (Run ``make`` in the dlopen directory to 238generate the manifest file if you haven't done so already.) Here is an excerpt 239from ``dlopen.nmf``:: 240 241 { 242 "files": { 243 "libeightball.so": { 244 "x86-64": { 245 "url": "lib64/libeightball.so" 246 }, 247 "x86-32": { 248 "url": "lib32/libeightball.so" 249 } 250 }, 251 "libstdc++.so.6": { 252 "x86-64": { 253 "url": "lib64/libstdc++.so.6" 254 }, 255 "x86-32": { 256 "url": "lib32/libstdc++.so.6" 257 } 258 }, 259 "libppapi_cpp.so": { 260 "x86-64": { 261 "url": "lib64/libppapi_cpp.so" 262 }, 263 "x86-32": { 264 "url": "lib32/libppapi_cpp.so" 265 } 266 }, 267 ... etc. 268 269In most cases, you can use the ``create_nmf.py`` script in the SDK to generate 270a manifest file for your application. The script is located in the tools 271directory (e.g. ``pepper_28/tools``). 272 273The Makefile in the dlopen example generates the manifest automatically using 274the ``NMF_RULE`` provided by the SDK build system. Running ``make V=1`` will 275show the full command line which is used to generate the nmf:: 276 277 create_nmf.py -o dlopen.nmf glibc/Release/dlopen_x86_32.nexe \ 278 glibc/Release/dlopen_x86_64.nexe glibc/Release/libeightball_x86_32.so \ 279 glibc/Release/libeightball_x86_64.so -s ./glibc/Release \ 280 -n libeightball_x86_32.so,libeightball.so \ 281 -n libeightball_x86_64.so,libeightball.so 282 283Run python ``create_nmf.py --help`` to see a full description of the command-line 284flags. A few of the important flags are described below. 285 286``-s`` *directory* 287 use *directory* to stage libraries (libraries are added to ``lib32`` and 288 ``lib64`` subfolders) 289 290``-L`` *directory* 291 add *directory* to the library search path. The default search path 292 already includes the toolchain and SDK libraries directories. 293 294.. Note:: 295 :class: note 296 297 **Note:** The ``create_nmf`` script can only automatically detect explicit 298 shared library dependencies (for example, dependencies specified with the -l 299 flag for the compiler/linker). If you want to include libraries that you 300 intend to dlopen() at runtime you must explcitly list them in your call to 301 ``create_nmf``. 302 303As an alternative to using ``create_nmf``, it is possible to manually calculate 304the list of shared library dependencies using tools such as ``objdump_``. 305 306Deploying a dynamically linked application 307========================================== 308 309As described above, an application's manifest file must explicitly list all the 310executable code modules that the application directly depends on, including 311modules from the application itself (.nexe and .so files), modules from the 312Native Client SDK (e.g., libppapi_cpp.so), and perhaps also modules from 313`naclports <http://code.google.com/p/naclports/>`_ or from 314`middleware systems <../../community/middleware>`_ that 315the application uses. You must provide all of those modules as part of the 316application deployment process. 317 318As explained in :doc:`Distributing Your Application 319<../distributing>`, there are two basic ways to deploy an application: 320 321* **hosted application:** all modules are hosted together on a web server of 322 your choice 323 324* **packaged application:** all modules are packaged into one file, hosted in 325 the Chrome Web Store, and downloaded to the user's machine 326 327You must deploy all the modules listed in your application's manifest file for 328either the hosted application or the packaged application case. For hosted 329applications, you must upload the modules to your web server. For packaged 330applications, you must include the modules in the application's Chrome Web 331Store .crx file. Modules should use URLs/names that are consistent with those 332in the Native Client manifest file, and be named relative to the location of 333the manifest file. Remember that some of the libraries named in the manifest 334file may be located in directories you specified with the -L option to 335``create_nmf.py``. You are free to rename/rearrange files and directories 336referenced by the Native Client manifest file, so long as the modules are 337available in the locations indicated by the manifest file. If you move or 338rename modules, it may be easier to re-run ``create_nmf.py`` to generate a new 339manifest file rather than edit the original manifest file. For hosted 340applications, you can check for name mismatches during testing by watching the 341request log of the web server hosting your test deployment. 342 343Opening a shared library at runtime 344=================================== 345 346Native Client supports a version of the POSIX standard ``dlopen()`` interface 347for opening libraries explicitly, after an application is already running. 348Calling ``dlopen()`` may cause a library download to occur, and automatically 349loads all libraries that are required by the named library. 350 351.. Note:: 352 :class: note 353 354 **Caution:** Since ``dlopen()`` can potentially block, you must initially 355 call ``dlopen()`` off your application's main thread. Initial calls to 356 ``dlopen()`` from the main thread will always fail in the current 357 implementation of Native Client. 358 359The best practice for opening libraries with ``dlopen()`` is to use a worker 360thread to pre-load libraries asynchronously during initialization of your 361application, so that the libraries are available when they're needed. You can 362call ``dlopen()`` a second time when you need to use a library -- per the 363specification, subsequent calls to ``dlopen()`` return a handle to the 364previously loaded library. Note that you should only call ``dlclose()`` to 365close a library when you no longer need the library; otherwise, subsequent 366calls to ``dlopen()`` could cause the library to be fetched again. 367 368The dlopen example in the SDK demonstrates how to open a shared libraries 369at runtime. To reiterate, the example includes three C++ files: 370 371* ``eightball.cc``: this is the shared library that implements the function 372 ``Magic8Ball()`` (this file is compiled into libeightball.so) 373* ``reverse.cc``: this is the shared library that implements the function 374 ``Reverse()`` (this file is compiled into libreverse.so) 375* ``dlopen.cc``: this is the Native Client module that loads the shared libraries 376 and makes calls to ``Magic8Ball()`` and ``Reverse()`` in response to requests 377 from JavaScript. 378 379When the Native Client module starts, it kicks off a worker thread that calls 380``dlopen()`` to load the two shared libraries. Once the module has a handle to 381the library, it fetches the addresses of the ``Magic8Ball()`` and ``Reverse()`` 382functions using ``dlsym()``. When a user types in a query and clicks the 'ASK!' 383button, the module calls ``Magic8Ball()`` to generate an answer, and returns 384the result to the user. Likewise when the user clicks the 'Reverse' button 385it calls the ``Reverse()`` function to reverse the string. 386 387Troubleshooting 388=============== 389 390If your .nexe isn't loading, the best place to look for information that can 391help you troubleshoot the JavaScript console and standard output from Chrome. 392See :ref:`Debugging <devcycle-debugging>` for more information. 393 394Here are a few common error messages and explanations of what they mean: 395 396**/main.nexe: error while loading shared libraries: /main.nexe: failed to allocate code and data space for executable** 397 The .nexe may not have been compiled correctly (e.g., the .nexe may be 398 statically linked). Try cleaning and recompiling with the glibc toolchain. 399 400**/main.nexe: error while loading shared libraries: libpthread.so.xxxx: cannot open shared object file: Permission denied** 401 (xxxx is a version number, for example, 5055067a.) This error can result from 402 having the wrong path in the .nmf file. Double-check that the path in the 403 .nmf file is correct. 404 405**/main.nexe: error while loading shared libraries: /main.nexe: cannot open shared object file: No such file or directory** 406 If there are no obvious problems with your main.nexe entry in the .nmf file, 407 check where main.nexe is being requested from. Use Chrome's Developer Tools: 408 Click the menu icon |menu-icon|, select Tools > Developer Tools, click the 409 Network tab, and look at the path in the Name column. 410 411**NaCl module load failed: ELF executable text/rodata segment has wrong starting address** 412 This error happens when using a newlib-style .nmf file instead of a 413 glibc-style .nmf file. Make sure you build your application with the glic 414 toolchain, and use the create_nmf.py script to generate your .nmf file. 415 416**NativeClient: NaCl module load failed: Nexe crashed during startup** 417 This error message indicates that a module crashed while being loaded. You 418 can determine which module crashed by looking at the Network tab in Chrome's 419 Developer Tools (see above). The module that crashed will be the last one 420 that was loaded. 421 422**/lib/main.nexe: error while loading shared libraries: /lib/main.nexe: only ET_DYN and ET_EXEC can be loaded** 423 This error message indicates that there is an error with the .so files listed 424 in the .nmf file -- either the files are the wrong type or kind, or an 425 expected library is missing. 426 427**undefined reference to 'dlopen' collect2: ld returned 1 exit status** 428 This is a linker ordering problem that usually results from improper ordering 429 of command line flags when linking. Reconfigure your command line string to 430 list libraries after the -o flag. 431 432.. |menu-icon| image:: /images/menu-icon.png 433.. _objdump: http://en.wikipedia.org/wiki/Objdump 434.. _GLIBC: http://www.gnu.org/software/libc/index.html 435.. _POSIX: http://en.wikipedia.org/wiki/POSIX 436.. _newlib: http://sourceware.org/newlib/ 437
