How to build from the Command Line
This is not a very difficult task really, since Lua comes with the command line tools to do so and in some cases installing add-ons are as simple as using 'luarocks' to do that for you. If you can tinker with the command line, then you know what you are doing. If you are struggling or stressed out because it doesn't work for you, read on...
This is applicable more so for Mac users than any other systems.
1. Tools
You will require the gcc package and it is part of the XCode commandline tools. It can also be downloaded from user kennethreitz's github repository at https://github.com/kennethreitz/osx-gcc-installer this saves you from downloading xcode just for gcc. While these are open-source tools, the (Apple version) sources can be found at http://www.opensource.apple.com/release/developer-tools-41/ and the gcc (binary build( installers can be chosen from the site https://github.com/kennethreitz/osx-gcc-installer/downloads based on your Mac OS X version.
2. Sources for the tools
While the source of most of the tools are found at the appropriate websites, LuaForge is one of the commonly used sources. However, let's take the example of sockets. I had sockets working fine for me and I followed an article from another user on installing luadocs via luarocks and BLAM... sockets stopped working. It is a bit of a shame that while the instructions make it all seem to simple and straight forward, it can result in things not working.
For the remainder of the post, we shall look at how to download, compile and install sockets on the Mac OS X.
We need wget or curl, these are two cmomandline utilities that allow us to download files from the commandline rather than from the browser. Apple has their own version of curl and the source can be downloaded from http://www.opensource.apple.com/source/curl/
you can also find compiled (binary) versions of curl ready to use from http://curl.haxx.se/download.html scroll down to the MAC OS X section or from http://www.hmug.org/pub/MacOS_X/BSD/Applications/Internet/curl/
Once you have curl, open terminal and type in
Next we need to download the socket luarock package,
now that we have downloaded the file, we need to expand it, the rock file is actually a compressed file, like a zip, but instead uses tar for compression/expansion.
We should now be in the source directory of luasocket, we need to make a few changes, you can look at the config file, by typing 'cat config' on the terminal or you can open it in text edit using 'open -e config'
If you read this file, it is in text, but can be cryptic for you, the issue is that when luaRocks installs the rock, it uses this config file to compile and install, all of this happens in the background. So you do not have any control on this process. This config file is if you see, geared for a linux environment, not Mac. So first thing we need to do is modify the config file accordingly.
Note, in the Linux system, the # is a comment, so all of the lines that start with a # are comments and we scroll to towards the bottom of the screen where there are two sets of instructions,
You would notice that the lines below the section for Mac OS X are all commented out and the ones under for Linux are all uncommented. We need to do the reverse, comment out all of the linux settings and uncomment the Mac OS X settings.
we will end up with something that looks like
the way to create / compile is to run the make command from the terminal. we can type
The 'make clean' removes all the compiled files and then compiles them again.
The Big Question
What is the problem, why are we doing this? What is the problem here? The problem is that while we allow gcc to compile, it creates the 64-bit version also called the 'Mach-O 64-bit bundle x86_64', what we really need is the 32-bit version which is called the 'Mach-O bundle i386'.
This little thing can lead to hours of frustration as there is no clear documentation on this issue and how to resolve it. You might be lucky to find some references, (as this is a very commonly asked question) that suggest using the '-m32' parameter. In some places you can find elaborate instructions that tell you that this needs to be added to the commandline while using gcc.
Let us see how gcc works, to do so, let us create a c file to compile.
this should compile without errors and if you type
you can see the two files, 'foo.c' and 'foo'. We need to get details on what type of file is this file 'foo', this is easily done by typing
and we see it returns
We want a 32-bit foo, how do we do that? We use the information we learned about using the -m32 as
and now if we type in 'file foo', it returns
Success, YAY!! but how do we apply this to the config file?
we add the -m32 to the line 'CC=gcc -m32' this will then use the -m32 flag for compiling all the files. Let's bring out the champagne, and celebrate... Not quite.
While this will compile, it will complain about the architecture being different. This can be a really stressful, if you had no idea on what's happening, this is the point when you just close everything and say, I can live without this stupid thing.
The problem is quite simple really and it is surprising why this is not mentioned commonly. We have compiled using gcc to a 32-bit compiled file, but when creating libraries, we need to link them this is where another component called the Linker comes into the picture. The command line utility is 'ld' and if you simply type in ld on the command line, it tells you
in our compiler and linker settings, we now add the line to specify the architecture as
and now if we run 'make clean; make all' it should work fine.
Sometimes these might require to be placed in the appropriate directories which are system ones like /usr/local, etc and access to these are not readily granted. This might result in permission errors. This can be resolved by issuing the
the sudo elevates the command to run as the administrator and would ask you for the password (the first time around).
Hope that was helpful in understanding the what's and the how's of the compilation hell from command line. Though the concept remains the same, sometimes it can be more difficult than this.
It is always good to compile your own binaries, you would be at rest that there is no malware added in comparison to downloading a binary from a website (unless of course it is reputable and the official source or you trust them). The alternatives are to get someone who knows to compile them for you or use another system to build like a Linux system or even (I cannot believe I am suggesting this) a windows box.
This is applicable more so for Mac users than any other systems.
1. Tools
You will require the gcc package and it is part of the XCode commandline tools. It can also be downloaded from user kennethreitz's github repository at https://github.com/kennethreitz/osx-gcc-installer this saves you from downloading xcode just for gcc. While these are open-source tools, the (Apple version) sources can be found at http://www.opensource.apple.com/release/developer-tools-41/ and the gcc (binary build( installers can be chosen from the site https://github.com/kennethreitz/osx-gcc-installer/downloads based on your Mac OS X version.
2. Sources for the tools
While the source of most of the tools are found at the appropriate websites, LuaForge is one of the commonly used sources. However, let's take the example of sockets. I had sockets working fine for me and I followed an article from another user on installing luadocs via luarocks and BLAM... sockets stopped working. It is a bit of a shame that while the instructions make it all seem to simple and straight forward, it can result in things not working.
For the remainder of the post, we shall look at how to download, compile and install sockets on the Mac OS X.
We need wget or curl, these are two cmomandline utilities that allow us to download files from the commandline rather than from the browser. Apple has their own version of curl and the source can be downloaded from http://www.opensource.apple.com/source/curl/
you can also find compiled (binary) versions of curl ready to use from http://curl.haxx.se/download.html scroll down to the MAC OS X section or from http://www.hmug.org/pub/MacOS_X/BSD/Applications/Internet/curl/
Once you have curl, open terminal and type in
curlif you get no error, everything is ok.
Next we need to download the socket luarock package,
curl http://luarocks.org/repositories/rocks/luasocket-2.0.2-5.src.rock > luasocket.src.rock
now that we have downloaded the file, we need to expand it, the rock file is actually a compressed file, like a zip, but instead uses tar for compression/expansion.
tar -zxf luasocket.src.rock tar -zxf luasocket-2.0.2.tar.gz cd luasocket-2.0.2
We should now be in the source directory of luasocket, we need to make a few changes, you can look at the config file, by typing 'cat config' on the terminal or you can open it in text edit using 'open -e config'
If you read this file, it is in text, but can be cryptic for you, the issue is that when luaRocks installs the rock, it uses this config file to compile and install, all of this happens in the background. So you do not have any control on this process. This config file is if you see, geared for a linux environment, not Mac. So first thing we need to do is modify the config file accordingly.
Note, in the Linux system, the # is a comment, so all of the lines that start with a # are comments and we scroll to towards the bottom of the screen where there are two sets of instructions,
#------ # Compiler and linker settings # for Mac OS X #and
#------ # Compiler and linker settings # for Linux
You would notice that the lines below the section for Mac OS X are all commented out and the ones under for Linux are all uncommented. We need to do the reverse, comment out all of the linux settings and uncomment the Mac OS X settings.
we will end up with something that looks like
#------ # Compiler and linker settings # for Mac OS X # CC=gcc DEF= -DLUASOCKET_DEBUG -DUNIX_HAS_SUN_LEN CFLAGS= $(LUAINC) -I$(COMPAT) $(DEF) -m32 -pedantic -Wall -O2 -fno-common LDFLAGS=-bundle -undefined dynamic_lookup LD=export MACOSX_DEPLOYMENT_TARGET="10.3"; gcc #------ # Compiler and linker settings # for Linux #CC=gcc #DEF=-DLUASOCKET_DEBUG #CFLAGS= $(LUAINC) $(DEF) -pedantic -Wall -O2 -fpic #LDFLAGS=-O -shared -fpic #LD=gcc
the way to create / compile is to run the make command from the terminal. we can type
make allthis will compile all the files that are required to be compiled for Luasocket. If it says nothing to do for all, you can start afresh by issuing
make clean make all
The 'make clean' removes all the compiled files and then compiles them again.
The Big Question
What is the problem, why are we doing this? What is the problem here? The problem is that while we allow gcc to compile, it creates the 64-bit version also called the 'Mach-O 64-bit bundle x86_64', what we really need is the 32-bit version which is called the 'Mach-O bundle i386'.
This little thing can lead to hours of frustration as there is no clear documentation on this issue and how to resolve it. You might be lucky to find some references, (as this is a very commonly asked question) that suggest using the '-m32' parameter. In some places you can find elaborate instructions that tell you that this needs to be added to the commandline while using gcc.
Let us see how gcc works, to do so, let us create a c file to compile.
cat > foo.c
main() {}
press 'control + D' to after you finish typing 'main() {}' this will save the file, now we can compile itgcc foo.c -o foo
this should compile without errors and if you type
ls -l foo
you can see the two files, 'foo.c' and 'foo'. We need to get details on what type of file is this file 'foo', this is easily done by typing
file foo
and we see it returns
foo: Mach-O 64-bit executable x86_64
We want a 32-bit foo, how do we do that? We use the information we learned about using the -m32 as
gcc -m32 foo.c -o foo
and now if we type in 'file foo', it returns
foo: Mach-O executable i386
Success, YAY!! but how do we apply this to the config file?
#------ # Compiler and linker settings # for Mac OS X # CC=gcc -m32 DEF= -DLUASOCKET_DEBUG -DUNIX_HAS_SUN_LEN CFLAGS= $(LUAINC) -I$(COMPAT) $(DEF) -m32 -pedantic -Wall -O2 -fno-common LDFLAGS=-bundle -undefined dynamic_lookup LD=export MACOSX_DEPLOYMENT_TARGET="10.3"; gcc
we add the -m32 to the line 'CC=gcc -m32' this will then use the -m32 flag for compiling all the files. Let's bring out the champagne, and celebrate... Not quite.
While this will compile, it will complain about the architecture being different. This can be a really stressful, if you had no idea on what's happening, this is the point when you just close everything and say, I can live without this stupid thing.
The problem is quite simple really and it is surprising why this is not mentioned commonly. We have compiled using gcc to a 32-bit compiled file, but when creating libraries, we need to link them this is where another component called the Linker comes into the picture. The command line utility is 'ld' and if you simply type in ld on the command line, it tells you
ld: warning: -arch not specified ld: warning: -macosx_version_min not specificed, assuming 10.7 ld: no object files specified for inferred architecture x86_64It will by default assume that the version of macosx is what your system is and the architecture is x86_64. which is the issue, it also gives us a hint that via the output '-arch not specified' this tells us that we can specify the architecture that we want.
in our compiler and linker settings, we now add the line to specify the architecture as
#------ # Compiler and linker settings # for Mac OS X # CC=gcc -m32 DEF= -DLUASOCKET_DEBUG -DUNIX_HAS_SUN_LEN CFLAGS= $(LUAINC) -I$(COMPAT) $(DEF) -m32 -pedantic -Wall -O2 -fno-common LDFLAGS=-bundle -undefined dynamic_lookup -arch i386 LD=export MACOSX_DEPLOYMENT_TARGET="10.3"; gcc
and now if we run 'make clean; make all' it should work fine.
Sometimes these might require to be placed in the appropriate directories which are system ones like /usr/local, etc and access to these are not readily granted. This might result in permission errors. This can be resolved by issuing the
sudo make clean sudo make install
the sudo elevates the command to run as the administrator and would ask you for the password (the first time around).
Hope that was helpful in understanding the what's and the how's of the compilation hell from command line. Though the concept remains the same, sometimes it can be more difficult than this.
It is always good to compile your own binaries, you would be at rest that there is no malware added in comparison to downloading a binary from a website (unless of course it is reputable and the official source or you trust them). The alternatives are to get someone who knows to compile them for you or use another system to build like a Linux system or even (I cannot believe I am suggesting this) a windows box.
Comments
Post a Comment