Creating and using static libraries in Linux

Static libraries are simply a collection of ordinary object files.

For more information on shared libraries checkout - Creating and using shared libraries in Linux

Static libraries conventionally end with the ".a" suffix. They are useful for developers to link to their library, but don't want to give the library source code. Theoretically code in static ELF libraries that is linked into an executable should run slightly faster (by 1-5%) than a shared library or a dynamically loaded library, but in practice this rarely seems to be the case due to other confounding factors.

We use following source code files for this post.

calc_mean.c

double mean(double a, double b)
{
return (a+b) / 2;
}

calc_mean.h

double mean(double, double);

main.c - We are including our static library in this application.

#include <stdio.h>
#include "calc_mean.h"

int main(int argc, char* argv[]) {

double v1, v2, m;
v1 = 5.2;
v2 = 7.9;

m  = mean(v1, v2);

printf("The mean of %3.2f and %3.2f is %3.2f\n", v1, v2, m);

return 0;
}

Creating the static library

First we have to create object file for calc_mean.c

gcc -c calc_mean.c -o calc_mean.o

Then, using archiver (ar) we produce a static library (named libmean.a) out of the object file calc_mean.o.

ar rcs libmean.a calc_mean.o

NOTE: A static library must start with the three letters 'lib' and have the suffix '.a'.

Compiling main program and linking with static library

We have already created a static library libmean.a and now let us use the static library by invoking it as part of the compilation and linking process when creating a program executable. Incase of gcc we use following flags to create static library

• -llibrary
searches for the library named library at the time of linking. Linker searches and processes libraries and object files in the order they are specified. Thus, ‘foo.o -lz bar.o’ searches library ‘z’ after file foo.o but before bar.o. If bar.o refers to functions in ‘z’, those functions may not be loaded. The linker searches a standard list of directories for the library, which is actually a file named liblibrary.a. The linker then uses this file as if it had been specified precisely by name.

• -L(path-to-the-library)
Specifies the path in which the library file can be found. We can use -L. inorder to point to the current directory and -L/home/tmp to point to /home/tmp directory.

• -static
On systems that support dynamic linking, this prevents linking with the shared libraries.

gcc -static main.c -L. -lmean -o main

Now run the executable program 'main'

$./main

ctags - vim with steroids

Basically C/C++ projects have thousands of lines of code divided into hundreds in some cases thousands of files. Inorder to access various function definitions within the sourcecode repository effectively using a VIM editor there is a great need for using addons like ctags which provides easy code go through.

Eventhough there are many effective GUI based code editors like eclipse e.t.c I prefer to use VIM editor as my primary code editor. I am not much into GUI funda so still prefer basic editor like VIM.

1. Installing ctags package
Almost all the linux flavours with 2.6.X kernel might have ctags installed by default. If not download the appropriate .deb or rpm file. Sorry I am not dealing with installing of ctags as I havent came across this stage as I am using FEDORA 9.

2. Generating ctags on your source code.
We have to generate a file named 'tags' for all the source code files, use the following command:

ctags *.c *.h

When we have many files in many directories then we have to create a tags file in each of the directories. But VIM will only be able to jump to tags within the same directory. To find more tags files, we have to set the 'tags' option in VIM to include all the relevant tags files. Just set the following command in ~/.vimrc file.

:set tags=./tags,./../tags,./*/tags

This finds a tags file in the same directory as the current file, one directory level higher and in all subdirectories.

But VIM searching many places for tags files is not really robust enough as it may get a bit slow. It's better to generate one big tags file offcourse it takes more time to create a single tags files for whole project. Just give the following command to recursively add all files in your project.

$ cd /proj 
$ ctags -R *

-R is to recursively go across all the directories, and a ‘tags’ file will be created including all the files in the sub-directories also.

Now we can simply include this single tags files as shown below

:set tags=~/proj/tags

3. Start using ctags with VIM editor
We have 3 different ways to use ctags in VIM editor.

From Shell:
We can invoke directly the file or the files containing the definition for a particular function from shell.

vi -t function_name

This will find the correct file or list of files having function_name definition.

OR

VIM command line:
We can invoke from VIM commandline (in command mode) the definition of a function.

:tag function_name
or
:ta function_name

This will jump automatically from the current file to the file containing the function_name definition.

OR

By cursor position:
This option is more user-friendly as we use a key combination instead of giving commands.

ctrl + ]

Place the cursor on the first character of the function name and press ctrl-]. This will jump to the file containing the definition of function_name.

ctrl + t

This command will jump back to previous location.

ctrl + i

To travel forward through the tag history.

ctrl + o

To travel backward through the tag history.

History
To display the list of tags that we have traversed in past give the following command.

:tags

Shows tag stack (history).

Divide and Conquer
As we saw already 'ctrl + ]' replaces the file in the current window with the one containing the new function. But suppose if we want to see not only the old function but also the new one?

We can split the window using the ":split" command followed by the ":tag" command.

:stag function_name

Cursor command for above feature is

ctrl + w + ]

Auto Complete
VIM supports tag name completion. Start typing the tag name (i.e. function name) and then hit TAB key and name completion will be done automatically if there is a tag name.

:tag start-of-tag-name_TAB

Jump to a tag name found by a search.

:tag /search-string

When multiple entries exist in the tags file, such as a function declaration in a header file and a function definition (the function itself), the operator can choose by issuing this command. The user will be presented with all the references to the function and the user will be prompted to enter the number associated with the appropriate one.

:tselect function-name

Jumps to next matching tag.

:tnext
or
:tn

Jump to previous matching tag.

:tprevious
or
:tp

Jump to first matching tag.

:tfirst
or
:tf
or
:trewindor:tr

Jump to last matching tag.

:tlast
or
:tl

Finding global identifiers
You are editing a C program and wonder if a variable is declared as "int" or "unsigned". A quick way to find this is with the "[I" command. Suppose the cursor is on the word "column" then type

[ + shiftkey + i

Vim will list the matching lines it can find. Not only in the current file,
but also in all included files (and files included in them, etc.). The result
looks like this:

 structs.h 
  1:   29     unsigned     column;    /* column number */ 

The advantage over using tags or the preview window is that included files are
searched. In most cases this results in the right declaration to be found.
Also when the tags file is out of date. Also when you don't have tags for the
included files.

However, a few things must be right for "[I" to do its work. First of all,
the 'include' option must specify how a file is included. The default value
works for C and C++. For other languages you will have to change it.

Reference:
ctags documentation

Creating Shared Libraries in Linux - Part 2

Check the PART1 of this article.

4. Making the library available at run-time

Using LD_LIBRARY_PATH
We have to create or set the environment variable "LD_LIBRARY_PATH" to the directory containing the shared libraries.

export LD_LIBRARY_PATH=/home/cf/lib

If in current directory you can give the following command

export LD_LIBRARY_PATH=.

If we have to append a new directory to the existing paths then add the directories separated by colons to environment variable "LD_LIBRARY_PATH".

export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH

Now recompile the main program

gcc -o test main.c -lcalc_mean -L/home/cf/slib

Now check the ldd command output

$ ldd test
 linux-gate.so.1 =>  (0x007ad000)
 libcalc_mean.so => ./libcalc_mean.so (0x0081e000)
 libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x005ff000)
 /lib/ld-linux.so.2 (0x00e19000)

It seems now linker is able to locate our shared library as we can see in above output. Now run the program

$./test

LD_LIBRARY_PATH is good for quick tests and for systems on which you don’t have admin privileges. As a downside, however, exporting the LD_LIBRARY_PATH environment variable  might screw up with other programs you run that also rely on LD_LIBRARY_PATH if you don’t reset it to its previous state when you’re done.

Creating and using shared libraries in Linux

What is a Shared Library ??

A library that is loaded into physical memory only once and reused by multiple processes via virtual memory. 

Generally Shared libraries are .so (or in windows .dll) files.

Why shared libraries ??

• They reduce memory consumption if used by more than one process, and they reduce the size of the executable.
• They make developing applications easier: a small change in the implementation of a function in the library don't need the user to recompile and relink his application code every time. You need to only relink if you make incompatible changes, such as adding arguments to a call or changing the size of a struct.

NOTE: Debugging using a shared library is slightly more difficult when compared with static libraries, because the debugger usually used on Linux, gdb, has some problems with shared libraries.

Let us see how to create a shared library on Linux. We use following source code files for this post.
calc_mean.h

#ifndef calc_mean_h__
#define calc_mean_h__
double mean(double, double);
#endif  // calc_mean_h__

calc_mean.c

double mean(double a, double b)
{
return (a+b) / 2;
}

main.c - We are including our shared library in this application.

#include <stdio.h>
#include "calc_mean.h"

int main(int argc, char* argv[]) {

double v1, v2, m;
v1 = 5.2;
v2 = 7.9;

m  = mean(v1, v2);

printf("The mean of %3.2f and %3.2f is %3.2f\n", v1, v2, m);

return 0;
}

1. Creating Object File with Position Independent Code
All the code that goes into a shared library needs to be position independent. We can make gcc emit position-independent code by passing it one of the command-line switches -fpic or -fPIC (the former is preferred, unless the modules have grown so large that the relocatable code table is simply too small in which case the compiler will emit an error message, and you have to use -fPIC).

First we will create object files for all .c files that goes into a shared library.

gcc -c -fPIC calc_mean.c -o calc_mean.o

Above we are compiling calc_mean.c with -fPIC option and generating calc_mean.o object file.

2. Creating Shared library with the Object File
Every shared library has a prefix "lib", the name of the library, the phrase ".so", followed by a period and a version number that is incremented whenever the interface changes (as a special exception, the lowest-level C libraries don't start with "lib").

gcc -shared -o libcalc_mean.so calc_mean.o

Above command on successful produces a shared library named "libcalc_mean.so".

• -shared: Produces a shared object which can then be linked with other objects to form an executable.

3. Using the Shared Library
Now let us link the created shared library with our application. Compile main.c as shown below

$ gcc -o test main.c -lcalc_mean
/usr/bin/ld: cannot find -lcalc_mean
collect2: ld returned 1 exit status

The linker doesn’t know where to find libcalc_mean. But why ?
GCC has a list of places to look by default for shared libraries, but our directory is not in that list. Bingo that's the reason compilation failed at linking level.

Now we need to tell GCC where to find libcalc_mean.so. We will do that with the -L option.

gcc -o test main.c -lcalc_mean -L/home/cf/slib

•  -l option tells the compiler to look for a file named libsomething.so The something is specified by the argument immediately following the “-l”. i.e. -lmean

• -L option tells the compiler where to find the library. The path to the directory containing the shared libraries is followed by "-L". If no “-L” is specified, the compiler will search the usual locations. "-L." means looking for the shared libraries in the current directory and "-L/home/cf/lib" means looking for the shared libraries at "/opt/lib" path. You can specify as many “-l” and “-L” options as you like.

NOTE: It would be a better idea to move all personal shared libraries in one directory rather in the current directory. For easy understanding I am moving "libcalc_mean.so" to "/home/cf/slib".

mv libcalc_mean.so /home/cf/slib

Now compile main.c. It would be successful and creates an executable named "test".
Let us check if the path to our shared library is included successfully into the executable by linker as shown below.

ldd executablename

$ ldd test 
 linux-gate.so.1 =>  (0x00332000)
 libcalc_mean.so => not found
 libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x006aa000)
 /lib/ld-linux.so.2 (0x00db9000)

You can see that linker cannot find our shared library libcalc_mean.so.

Basically libraries are present in /usr/lib amongst other places. Static libraries (.a suffix) are incorporated into the binary at link time, whereas dynamic ones (.so suffix) are referenced by location.

Check the PART2 of this article to understand further

Using 'find' command

'find' is one of the useful commands available in Unix/Linux systems. In this article I am trying to use 'find' in a effective way.

Basic syntax of find command as per man page is - find [path...] [expression]

NOTE: There are many options available for find command which are available in man page for find. Only some of those options are shown in this tutorial.

Finding a particular file in your system

$ find / -name 'filename' 2>/dev/null
$ find / -name 'filename' 2>errors.txt

/ - Start searching from the root directory (i.e / directory)

-name - Given search text is the filename rather than any other attribute of a file 'filename'. Always enclose the filename in single quotes...

NOTE: 2>/dev/null is not related to find tool as such. 2 indicates the error stream in Linux, and /dev/null is the device where anything you send simply disappears. So 2>/dev/null in this case means that while finding for the files, in case any error messages pop up simply send them to /dev/null i.e. simply discard all error messages.

Alternatively you could use 2>error.txt where after the search is completed you would have a file named error.txt in the current directory with all the error messages in it.

$ find -name 'met*'

The above command would start searching for the files that begin with the letters 'met' within the current directory and the directories that are present within the current directory.

If no paths are given, the current directory is used. If no expression is given, the expression ‘-print’ is used.

Searching with respect to type of the file (-type)

$find . -name 'temp'
./keepout/temp
./temp

$find . -name 'temp' -type d
./temp

Where d - directory, p - named pipe (FIFO), f - regular file and so on

Ignoring case-sensitivity (-iname)

$ find /home/temp -iname 'index*'

This command searchs for a file starting with string 'index' without considering the case of the filename. So all files starting with any combination of letters in upper and lower case such as INDEX or indEX or index would be returned.

Searching for a file based on size and time

$ find /home/songs -name '*.mp3' -size -5000k

$ find / -size +10000k

First command finds within a directory called /home/songs, only those mp3 files that have a size less than 5000 Kilobytes.

$ find /home/temp -amin -10 -name '*.c'
$ find /home/temp -atime -2 -name '*.c'
$ find /home/temp -mmin -10 -name '*.c'
$ find /home/temp -mtime -2 -name '*.c'

The 1st command searches for those files that are present in the directory /home/temp and its subdirectories which end in .c and which have been accessed in the last 10 minutes.

The 2nd command does the same but searches for those files that have been accessed in the last 10 hours.

The 3rd and the 4th commands do the same as the 1st and 2nd commands but they search for modified files rather than accessed files. Only if the contents of the files have been modified, would their names be returned in the search results.

$ find / -mount -name 'win*'

This command searches for files starting with the letters 'win' in their filenames. The only difference is that the mounted filesystems would not be searched for this time. This is useful when you have your Windows partitions mounted by default. And a search for 'win' might return many files on those partitions, which you may not be really interested in. This is only one use of -mount parameter.

$ find /home/songs -name 'Metallica*' -and -size +10000k
$ find /home/songs -size +10000k ! -name "Metallica*"
$ find /home/songs -name 'Metallica*' -or -size +10000k

Boolean operators such as AND, OR and NOT make find an extremely useful tool.

The 1st command searches within the directory /songs for files that have their names beginning with 'Metallica' and whose size is greater than 10000 kilobytes (> 10 MB).

The 2nd command searches in the same directory as above case but only for files that are greater than 10MB, but they should not have 'Metallica' as the starting of their filenames.

The 3rd command searches in the same directory for files that begin with 'Metallica' in their names or all the files that are greater than 10 MB in size.

How to apply a unix command to a set of files (-exec) ?

$ find . -name '*.sh' -exec chmod o+r '{}' \; -print

This command will search in the current directory and all sub directories. All files ending with .sh extension will be processed by the chmod -o+r command. The argument '{}' inserts each found file into the chmod command line. The \; argument indicates the exec command line has ended.

The end results of this command is all .sh files have the other permissions set to read access (if the operator is the owner of the file).

Searching for a string in a selection of files (-exec grep ...).

$ find . -exec grep "hello" '{}' \; -print

Prints all files that contain the string 'hello' will have their path printed to standard output.

If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrance of the search string. It then signals success to find and find continues searching for more files.

find . -exec grep -q "hello" '{}' \; -print

Finding Empty files (-empty)

$find . -empty

To delete empty files in the current directory:

$ find . -empty -maxdepth 1 -exec rm '{}' \;

For more examples try out

1. linux.ie
2. Devdaily
3. hccfl.edu