77 replies to this topic

[securitybreach]

From the Archlinux news:

The /lib directory becomes a symlink

Quote

2012-07-14

All Arch Linux packages have had their files in the /lib directory moved to /usr/lib and now /lib is a symlink to usr/lib. When performing this update, pacman will likely identify a conflict in the /lib directory. In the simplest case, this is worked around by doing:

pacman -Syu --ignore glibc
pacman -Su

Never use --force during this update.
If either of this steps does not work (e.g. due to dependency version issues, file conflicts in /lib), refer to this guide for more detailed instructions on performing this upgrade.

[Vagabond]

EDIT:  Figured out where I messed up.  No problem now.  

Ignore this stuff!



1.  # pacman -Syu --ignore glibc

Went fine.


2.  # pacman -Su

No good: /lib exists in filesystem.  No dependency errors
reported, so I moved on to "Issue 2" in the ArchWiki entry.


3.  I removed several directories in /lib that contained outdated
stuff.  This left /lib with no directories in it.  All remaining files are .so
or .so1 or .so2.

Rebooted.

# pacman -Su

"...conflicting files.../lib exists in filesystem".


4.  Checking for symlink status of /lib:

# ls -ld /lib

Told me things were not right:

drwxr-xr-x  2  root  root  4096  Jul14  10:49  /lib


5.  Info from Arch Newbie Forum: "Installed packages owning files in
/lib have to be uninstalled and re-installed later."  What are those files?

# find /lib -exec pacman -Qo -- {} +

Response:

A.  Can not determine ownership of /lib
B.  Everything within /lib is owned by glibc


6.  Which packages on my system think they own /lib?

# grep '^lib/' /var/lib/pacman/local/*/files

This gives a long list of files such as...

/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libBrokenLocale-2.16.so

...and...

/var/lib/pacman/local/nouveau-firmware-20091212-4/files:lib/firmware/nouveau/nv44.ctxprog

All the nouveau-firmware stuff was deleted from /lib.  All that
remains in /lib now is owned by glibc.


7.  I do not have anything "nouveau" or "firmware" installed; I use
the nVidia driver for my graphics card.

# pacman -Syu

"Conflicting files -- /lib exists in filesystem".


Another opportunity to learn something!  TIA

Edited by Vagabond, 14 July 2012 - 10:37 PM.

[V.T. Eric Layton]

Using yaourt, I think everything went fine. Rebooting now to test...

[V.T. Eric Layton]

OK, got a few module loading errors at boot, but all seems to be working normally. I'll check the error logs tomorrow for details.

G'night all...

[securitybreach]

Well I started out following the News information but I was having issues with the Catalyst-total package and the old modules from previous kernels. Before work I had a broken installation with a messed up ati driver ( 4th monitor was not working). Well after reading your fixes, I removed all of the old modules and also removed the Catalyst-total package. I then reinstalled the catalyst package, rebooted and restored my /etc/ati and /etc/X11 directories that I had previous backed up to github and then rebooted again. Well this time around, I have a better monitor placement than I had before due to today's catalyst update. I have my monitors set up with my main monitor on the bottom and then three other monitors above the main one. Well before it would treat them all as being in a single row which worked but was not perfect like it was on my windows gaming platform. After today's catalyst update, now all of the monitors are setup exactly like they are physically. This is how it looks physically:



I can move my pointer up and it actually goes to the monitor above my main monitor.

Now if I can just get Ian (ichase) to provide a tutorial on how he clones his system as I was scared that I lost everything. I have my main config files synced to github but that is not a true clone.


Thanks again for your writeup as the modules part really helped me out a lot!!!

[ichase]

Well I cloned the partition prior to starting down this road.  Jacked up my Arch and re-imaged.  Now I am still having issues.  I have been following the Wiki and up to this point I am getting a lot of stuff owned by glibc

[root@archbox ichase]# grep '^lib/' /var/lib/pacman/local/*/files
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/ld-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/ld-linux.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libBrokenLocale-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libBrokenLocale.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libSegFault.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libanl-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libanl.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libc-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libc.so.6
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libcidn-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libcidn.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libcrypt-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libcrypt.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libdl-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libdl.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libm-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libm.so.6
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libmemusage.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnsl-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnsl.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_compat-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_compat.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_db-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_db.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_dns-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_dns.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_files-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_files.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_hesiod-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_hesiod.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_nis-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_nis.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_nisplus-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libnss_nisplus.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libpcprofile.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libpthread-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libpthread.so.0
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libresolv-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libresolv.so.2
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/librt-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/librt.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libthread_db-1.0.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libthread_db.so.1
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libutil-2.16.so
/var/lib/pacman/local/glibc-2.16.0-1/files:lib/libutil.so.1
/var/lib/pacman/local/ld-lsb-3-3/files:lib/
/var/lib/pacman/local/ld-lsb-3-3/files:lib/ld-lsb.so.3

Not sure how to proceed at this point as the Wiki states

Quote

These packages need rebuilding so as not to include the /lib directory. Then the final "pacman -Su" will successfully install glibc.

Looks like a lot of packages to rebuild?

I will put an updated Clonezilla tutorial shortly.  Now that you have everything up and running Josh especially much better than what you had before, I HIGHLY recommed having a clone as a few hours ago, I cloned then re-imaged with no issue.  Back to where I was just a few hours ago.

I know Frank Golden put together a tutorial for Clonezilla which resides in the tutorials section of Bruno's classroom.  I was actually planning on putting together one with screen shots and asking if the admins would want it, but did not know how to approach it with Frank being a well respected member of BATL for a long time and the fact that he unfortunantly is no longer with us.

[securitybreach]

ichase, on 15 July 2012 - 02:21 AM, said:

Well I cloned the partition prior to starting down this road.  Jacked up my Arch and re-imaged.  Now I am still having issues.  I have been following the Wiki and up to this point I am getting a lot of stuff owned by glibc
Not sure how to proceed at this point as the Wiki states

Looks like a lot of packages to rebuild?

I will put an updated Clonezilla tutorial shortly.  Now that you have everything up and running Josh especially much better than what you had before, I HIGHLY recommed having a clone as a few hours ago, I cloned then re-imaged with no issue.  Back to where I was just a few hours ago.

I know Frank Golden put together a tutorial for Clonezilla which resides in the tutorials section of Bruno's classroom.  I was actually planning on putting together one with screen shots and asking if the admins would want it, but did not know how to approach it with Frank being a well respected member of BATL for a long time and the fact that he unfortunantly is no longer with us.

Did you get to this part?

Quote

Issue 2: The final "pacman -Su" still has conflicts in /lib

This means that you have files or folders still in /lib or pacman thinks a package apart from glibc still own /lib. You can see which package own files in /lib by using:
find /lib -exec pacman -Qo -- {} +

If any package apart from glibc is listed as owning a file, that package needs to be updated to install its files in /usr/lib. Any files unowned by a package should either be deleted or moved to /usr/lib and any directories within /lib need deleted (after they are empty...)
One common source of files in /lib is left overs from running depmod on a kernel update. For example:
error: No package owns /lib/modules/3.1.9-2-ARCH/modules.*

If no other files are in /lib/modules, you can safely remove that directory.

As far as a visual tutorial, I think that would be wonderful and I am pretty sure that Frank would agree with me. If anything, I think that would just be a nice addition to Frank's tutorial.

[ichase]

Yes, I got rid of the directories in /lib/modules then running # find /lib -exec pacman -Qo --{} + get me this:

error: cannot determine ownership of directory '/lib'
/lib/libnss_nis.so.2 is owned by glibc 2.16.0-1
/lib/libdl.so.2 is owned by glibc 2.16.0-1
/lib/ld-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_compat-2.16.so is owned by glibc 2.16.0-1
/lib/libanl.so.1 is owned by glibc 2.16.0-1
/lib/libc-2.16.so is owned by glibc 2.16.0-1
/lib/libcrypt.so.1 is owned by glibc 2.16.0-1
/lib/libpcprofile.so is owned by glibc 2.16.0-1
error: cannot determine ownership of directory '/lib/modules'
/lib/libpthread.so.0 is owned by glibc 2.16.0-1
/lib/ld-linux.so.2 is owned by glibc 2.16.0-1
/lib/libm-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_nisplus-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_dns.so.2 is owned by glibc 2.16.0-1
/lib/libnss_files-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_hesiod-2.16.so is owned by glibc 2.16.0-1
/lib/libutil.so.1 is owned by glibc 2.16.0-1
/lib/libSegFault.so is owned by glibc 2.16.0-1
/lib/ld-lsb.so.3 is owned by ld-lsb 3-3
/lib/libnss_db.so.2 is owned by glibc 2.16.0-1
/lib/libpthread-2.16.so is owned by glibc 2.16.0-1
/lib/libanl-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_hesiod.so.2 is owned by glibc 2.16.0-1
/lib/libdl-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_files.so.2 is owned by glibc 2.16.0-1
/lib/libcidn.so.1 is owned by glibc 2.16.0-1
/lib/libcidn-2.16.so is owned by glibc 2.16.0-1
/lib/librt-2.16.so is owned by glibc 2.16.0-1
/lib/libthread_db.so.1 is owned by glibc 2.16.0-1
/lib/libBrokenLocale-2.16.so is owned by glibc 2.16.0-1
/lib/libmemusage.so is owned by glibc 2.16.0-1
/lib/libnss_nis-2.16.so is owned by glibc 2.16.0-1
/lib/libutil-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_dns-2.16.so is owned by glibc 2.16.0-1
/lib/libnss_nisplus.so.2 is owned by glibc 2.16.0-1
/lib/libthread_db-1.0.so is owned by glibc 2.16.0-1
/lib/libnsl-2.16.so is owned by glibc 2.16.0-1
/lib/libm.so.6 is owned by glibc 2.16.0-1
/lib/libresolv-2.16.so is owned by glibc 2.16.0-1
/lib/libc.so.6 is owned by glibc 2.16.0-1
/lib/libresolv.so.2 is owned by glibc 2.16.0-1
/lib/libnss_db-2.16.so is owned by glibc 2.16.0-1
/lib/libnsl.so.1 is owned by glibc 2.16.0-1
/lib/librt.so.1 is owned by glibc 2.16.0-1
/lib/libnss_compat.so.2 is owned by glibc 2.16.0-1
/lib/libBrokenLocale.so.1 is owned by glibc 2.16.0-1
/lib/libcrypt-2.16.so is owned by glibc 2.16.0-1

I have gotten a lot better with Arch linux over the past year, but there is some things that I am still in the dark on.  Getting though this is obviously one of them as the explanation in the Wiki must still be over my head at my Arch linux experience level.  

On a seperate note, I will start working on the clonezilla screen shot tutorial.  May be a couple of days for me to get it together but I think I can put something really good together in conjunction with what Frank has already done.  Frank taught me how to use Clonezilla so I will make him proud.  

[securitybreach]

I am out with my cousin till this afternoon but I will look into it then.  Btw I think that you are fine because the wiki was talking about apps not libs. Those are normal, my problem was the catalyst package. It will easy to fix as no other apps are conflicting on your machine.

[ichase]

securitybreach, on 15 July 2012 - 12:09 PM, said:

I am out with my cousin till this afternoon but I will look into it then.  Btw I think that you are fine because the wiki was talking about apps not libs. Those are normal, my problem was the catalyst package. It will easy to fix as no other apps are conflicting on your machine.

So so far everything seems to be running fine, but I still have to install the update to glibc at some point, or do I?

[securitybreach]

Yes, you will need to update glibc as lots of packages depend on it. We will get you sorted when I get home.

[V.T. Eric Layton]

Ian, start a new thread for your Clonezilla tutorial. I'll move it and lock it after you're finished. Please add a note at the bottom asking folks to NOT post on the thread, just in case someone tries to before I get a chance to lock/move it.

Frank wouldn't mind at all.

[V.T. Eric Layton]

I used the steps at this wiki to solve the errors when attempting to update glibc. All's well now.

[securitybreach]

ichase, on 15 July 2012 - 12:03 PM, said:

Yes, I got rid of the directories in /lib/modules then running # find /lib -exec pacman -Qo --{} + get me this:

Have you tried running the commands again after clearing out the directories? Since you have your setup cloned, try running this:

pacman -Syu --ignore glibc
rm -rf /lib/modules
pacman -Syu

Reference: https://bbs.archlinu...php?pid=1130105

BTW that is what I did

[V.T. Eric Layton]

Ooops! I just noticed that I forgot to link to the dev's wiki that I mentioned above. Here it is...

https://wiki.archlin...operWiki:usrlib

[securitybreach]

V.T. Eric Layton, on 15 July 2012 - 08:19 PM, said:

Ooops! I just noticed that I forgot to link to the dev's wiki that I mentioned above. Here it is...

https://wiki.archlin...operWiki:usrlib

[Neil P]

I did this, and it worked fine (as far as I can tell...) but I have no idea why I did it. Why symlink everything? Why ignore glibc first? What did the first update do? For that matter, what did the second one do?

[securitybreach]

Neil P, on 15 July 2012 - 11:14 PM, said:

I did this, and it worked fine (as far as I can tell...) but I have no idea why I did it. Why symlink everything? Why ignore glibc first? What did the first update do? For that matter, what did the second one do?

Well I do not know the reason but "all files in the /lib directory have been moved to /usr/lib and now /lib is a symlink to usr/lib."  I imagine the symlink is there for packages that have not updated to reflect the latest glibc changes.

As far the ignoring glibc command, I think that is so that you can update the required files (kernel, etc) that depend on the newest glibc version. The second command, pacman -Su, just performs an update without resyncing the mirrors first.

Ref: http://mailman.archl...uly/023200.html

[Neil P]

Interesting. The symlink part is about the only one I understand. Just not why they did it. Anyway, it says not to use --force. What happens in this case if you force it?

[securitybreach]

Neil P, on 15 July 2012 - 11:59 PM, said:

Interesting. The symlink part is about the only one I understand. Just not why they did it.

Well this is due to an upstream change in the glibc package. Archlinux just gave instruction on how to update it without breaking your system. Blame the glibc devs

Neil P, on 15 July 2012 - 11:59 PM, said:

Anyway, it says not to use --force. What happens in this case if you force it?

You will break your Archlinux installation as a lot of packages depend on glibc including the kernel.

[sunrat]

I'm interested to find out why this is happening too. I always thought /lib was for system libraries and /usr/lib for user app libraries. The distinction is a little blurry though, just looking now (aptosid) and gcc and hal are in /usr/lib and systemd and udev are in /lib.

[securitybreach]

sunrat, on 16 July 2012 - 02:52 AM, said:

I'm interested to find out why this is happening too. I always thought /lib was for system libraries and /usr/lib for user app libraries.

Yeah so do I

That said, the problem is that glibc is a major part of the system.It also contains links which loads shared libraries for applications. If you move it carelessy, it will cause problems. Hence the caution of using the -f (force) switch.

[securitybreach]

More references: http://mailman.archl...rch/022625.html

[sunrat]

securitybreach, on 16 July 2012 - 11:10 AM, said:

More references: http://mailman.archl...rch/022625.html

So essentially, /lib /bin were there for libraries that were needed early in boot and may have been on a separate filesystem, and systemd makes this unnecessary? Applying KISS principle.
Funny that not long ago it was deemed necessary to move /var/run to a newly created /run directory. I'm sure there are good reasons.

[V.T. Eric Layton]

securitybreach, on 16 July 2012 - 11:10 AM, said:

More references: http://mailman.archl...rch/022625.html

That's an interesting read, Josh. Initially, I thought this change may have something to do with the Linux Foundations's upcoming FHS (Filesystem Hierarchy Standard) 3.0 coming out soon. The Linux Foundation maintains the Linux FHS. I'm kinda' surprised that Arch devs are taking it upon themselves, seemingly, to change the FHS in Arch to a non-standard system. Sure, diversity is a good thing, but I believe that GNU/Linux does need to stick to some standards; without which it could all devolve into a giant cluster... well, you know.