Dao's BLOG

Sometimes your NFS mount stopped response. You cannot umount normally, if you use umount -l to unmount it, you cannot mount it again, Mounting attempt stalls too. The suggestion on the interenet was to just reboot the client. But you may have important jobs or services on it and you really do not want to.

Here is a rescue:

After you unmounted the volume using umount -l, edit your /etc/fstab file, add vers=3 in front of the “defaults,_netdev” or whatever nfs mounting option column of the stalled volume, save it. Then you try mounting it again, and it is mounted.

This is not the end of story. There are some important new features from NFS_v4 that you do not want to permanently backup to NFS_v3. And someday the vers=3 mount may stall again.

Do not worry. Actually as long as the nfs server is still working, the calls that caused nfs stall will resolve or time out slowly. Wait a day or two, then you go back to the nfs client node, umount the volume mounted with vers=3. If it is in use, use umount -l to unmount it. Then you go editing the /etc/fstab file again, remove the vers=3 instruction, save it. Then you go back to your prompt and mount the volume again. As long as all pending calls that caused the original nfs stall are resolved, you will be able to mount the volume by default version, NFS_v4, again.

This process can be repeated is similar issue happens again, and you are free of being forced to reboot the nfs client box.

Sometimes when an dnf update was unexpectedly terminated like a power failure, the system may leave an inconsistent rpm database. If this happens, in most case by rebuilding rpm database (rpmdb –rebuilddb), and to use (package-cleanup –cleandupes), it could be fixed.

For some severe case, after both, when you are trying to run (dnf update –allowerasing), it still shows this kind of errors:

Running transaction check
error: rpmdbNextIterator: skipping h# ***
Header V3 RSA/SHA1 Signature, key ID ********: BAD
Header SHA256 digest: OK
Header SHA1 digest: OK

And the install/update cannot go through.

If you use (rpm -qa | grep <affected-package>), it shows that the package is not installed. Reinstalling the package using (rpm -Uvh <downloaded package name>) does not help.

This means that a package was erased in the interrupted dnf install/update but the record was not removed from the rpm database. You will need to do

rpm -e –justdb –noscripts –nosignature <package name>

to remove it.

However, sometimes the blocking package name cannot be found in the rpm database.

Here is a clue: when you are doing dnf install, it spits off errors like

Error: transaction check vs depsolve:
libxxx.so.x()(64bit) is needed by <package name>

This shows what lib file is missing.

You can go to internet to google the rpm package of the libxxx.so.x to find which package for your linux distro provides the missing lib file. Then run the

rpm -e –justdb –noscripts –nosignature <package name>

to remove the missing package.

After this, the dnf should function properly thereafter.

If ForwadX11 and ForwardX11Trusted are both enabled in /etc/ssh/ssh_config, when you do “ssh host” it will invoke X11 forwarding by default. This is convenient when you frequently need to run programs with GUI on remote host. However, with X11 forwarding, if you are simply running a text mode command/script via ssh, like “ssh node ls” to list files on a remote host, it may not exit properly with X11 forwarding.

To avoid this, the simplest way is to disable X11 forwarding in the ssh command using -x option: using “ssh -x node command” instead of “ssh node command”.

echo <your password> | ssh <node name> “sudo -S <command>”

For a RAID NAS box, when the NAS box dies, the data on the drives are still intact. They can be retrieved in the following steps:

1. dd disks containing LVM to hard drive; # create images of hard drives
2. losetup /dev/loopx disks.img; # make images appear as loop devices on OS
3. partx -v -add /dev/loopx; # make partitions on the loopback devices available to OS
4. vgscan, vgimport -a; # to import Virtual Groups
5. vgdisplay; # showing imported VG names
6. vgchange -a y vgname; # to activate VG
7. fsck /dev/mapper/vgname; # to check file system of VG
8. fuseext2 -o ro -o sync_read /dev/mapper/vgname /mounting_point; # to mount the VG
9. rsync -av –progress /mounting_point/ /destination/

Then you will get everything at your destination, and then you can safely remove the images.

Sometimes a bash script does not run the commands as cron or start up scripts that usually run when you debugging it. This may be caused by that the embedded running environment does not provide proper search path. To avoid this, always use full path to the executibles.

update.packages(checkBuilt=TRUE, ask=FALSE)

The convenient select copy, mouse middle click paste way does not work on some Fedora desktop environment. Even the Ctrl-C/Ctrl-V/Ctrl-X copy past and menu copy-paste does not work, forcing users to hand type in things that usually just a few mouse clicks away.

This is identified as a buggy package called “clipit”

To identify the problem, please do

$ ps ux | grep clipit

If it shows up like

username       1704  0.1  0.1 290772  5980 ?        Sl   13:04   0:51 clipit

You suffer this problem. To make it instantly working, please kill it:

$ kill <PID>

Then, to permanently remove it, you may simply uninstall it:

$ dnf erase clipit

and allow all packages that depends on it to be removed too.

Linux search /etc/profile.d and ~/.bashrc, ~/.bash_profile for system and user defined environmental variables. One of the most manipulated variable is $PATH. You can see this by running

$ echo $PATH

In most cases, when a new 3rd party software is installed, and you want it to be in your default executable search path, you insert a line like

export PATH=$PATH:/where/your/new_executables

in your ~/.bash_profile or in a file in /etc/profile.d if you are a system administrator. This appends a new search path onto the existing one.

However, if you launch another terminal, a shell, etc., all these scripts are loaded again and you got duplicated entries in $PATH.

This looks harmless and does not affect system performance in most cases since bash looks for the first available executable in the search path and ignore all later appended paths.

However, if you are running some scripts that loop into multiple layers of shells, this could cause the $PATH string overflow and disrupt your script. Also for cosmetic reason, a very long $PATH filled with duplicates are ugly.

To not generate duplicates, we can do:

NEWPATH=/where/your/new_executables 
if [ -d "$NEWPATH" ] && [[ ":$PATH:" != *":$NEWPATH:"* ]]; then
    PATH="${PATH:+"$PATH:"}$NEWPATH"
fi
export PATH;

for appending or

NEWPATH=/where/your/new_executables 
if [ -d "$NEWPATH" ] && [[ ":$PATH:" != *":$NEWPATH:"* ]]; then
    PATH="$NEWPATH${PATH:+":$PATH"}"
fi
export PATH;

for prepending, instead of the

export PATH=$PATH:/where/your/new_executables

way. This, will prevent generating duplicated entries in $PATH since it will only add in a new entry when the new entry is not found in the existing environmental variable.

For CentOS7,

ccmake -DCMAKE_INSTALL_PREFIX=/usr/local/lapack -DCMAKE_BUILD_TYPE=RELEASE -DBUILD_SHARED_LIBS=ON -DLAPACKE=ON ..

Then make, make install will get you a working version of liblapacke.so

The current version 3.8.0 source has bugs that prevent the .so to work properly. Version 3.7.1 is the best to do.

For CentOS8, the Fedora 29 version of Lapack works by dnf install.