I have a container file that uses the base image of fedora 37. I want to install some python libraries and some of them require gssapi. I know that gssapi requires krb5-devel. I have a step to install that and I can confirm that it works by calling krb5-config. However, while building the container (using Kaniko) it hit the issue of incompleted installation of gssapi because krb5-config was not found.
I could build my container fine locally using podman/buildar but when pushing my code to GitLab and using Kaniko to build the container, it failed. I doubt if it is because of Kaniko.
Here is my container file:
FROM fedora:37
USER root
WORKDIR /opt/workspace
RUN dnf install -y createrepo_c \
krb5-devel \
python3-pip \
python3-devel \
tree \
diffutils \
git \
gcc
RUN krb5-config
COPY *requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt -r test_requirements.txt
I also tried to install heimdal-devel and did the symblink RUN ln -s /usr/bin/heimdal-krb5-config /usr/bin/krb5-config but it doesn't help at all.
I have a JSON configuration for my pipeline in Pachyderm:
{
"pipeline": {
"name": "mopng-beneficiary-v2"
},
"input": {
"pfs": {
"repo": "mopng_beneficiary_v2",
"glob": "/*"
}
},
"transform": {
"cmd": ["python3", "/pclean_phlc9h6grzqdhm6sc0zrxjne_UdOgg.py /pfs/mopng_beneficiary_v2/euoEQHIwIQTe1wXtg46fFYok.csv /pfs/mopng_beneficiary_v2//Users/aviralsrivastava/Downloads/5Feb18_master_ujjwala_latlong_dist_dno_so_v7.csv /pfs/mopng_beneficiary_v2//Users/aviralsrivastava/Downloads/ppac_master_v3_mmi_enriched_with_sanity_check.csv /pfs/mopng_beneficiary_v2/Qc.csv"],
"image": "mopng-beneficiary-v2-image"
}
}
And my docker file is as follows:
FROM ubuntu:14.04
# Install opencv and matplotlib.
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y unzip wget build-essential \
cmake git pkg-config libswscale-dev \
python3-dev python3-numpy python3-tk \
libtbb2 libtbb-dev libjpeg-dev \
libpng-dev libtiff-dev libjasper-dev \
bpython python3-pip libfreetype6-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt
RUN sudo pip3 install matplotlib
RUN sudo pip3 install pandas
# Add our own code.
ADD pclean.py /pclean.py
However, when I run my command to create the pipeline:
pachctl create-pipeline -f https://raw.githubusercontent.com/avisrivastava254084/learning-pachyderm/master/pipeline.json
The files are existing in the pfs:
pachctl put-file mopng_beneficiary_v2 master -f /Users/aviralsrivastava/Downloads/pclean_phlc9h6grzqdhm6sc0zrxjne_UdOgg.py
➜ ~ pachctl put-file mopng_beneficiary_v2 master -f /Users/aviralsrivastava/Downloads/5Feb18_master_ujjwala_latlong_dist_dno_so_v7.csv
➜ ~ pachctl put-file mopng_beneficiary_v2 master -f /Users/aviralsrivastava/Downloads/ppac_master_v3_mmi_enriched_with_sanity_check.csv
➜ ~ pachctl put-file mopng_beneficiary_v2 master -f /Users/aviralsrivastava/Downloads/euoEQHIwIQTe1wXtg46fFYok.csv
It should be worth to note that I am getting this from the logs command(pachctl get-logs --pipeline=mopng-beneficiary-v2):
container "user" in pod "pipeline-mopng-beneficiary-v2-v1-lnbjh" is waiting to start: trying and failing to pull image
As Matthew L Daniel commented, the image name looks funny because it has no prefix. By default, Pachyderm pulls Docker images from Dockerhub, and Dockerhub prefixes images with the user that owns them (e.g. maths/mopng-beneficiary-v2-image)
Also, I think you might need to change the name of your input repo to be more distinct from the name of the pipeline. Pachyderm canonicalized repo names to meet Kubernetes naming requirements, and mopng-beneficiary-v2 and mopng_beneficiary_v2 might canonicalize to the same repo name (you might be getting an error like repo already exists). Try renaming the input repo to mopng_beneficiary_input or some such
I'm trying to build from source MariaDB on CentOS 7. I have only done the following commands before I get the error shown in this image:
$ cd /usr/local/src
$ wget https://downloads.mariadb.org/interstitial/mariadb-5.5.40/source/mariadb-5.5.40.tar.gz/from/http%3A//ftp.ddg.lth.se/mariadb
$ tar xzvf mariadb
$ cd mariadb-5.5.40/
$ chown -R root.root .
$ cmake . -DCMAKE_INSTALL_PREFIX:PATH=/usr/local/mariadb
proof that libaio-devel is installed
For anyone that is in a similar situation, cmake apparently does a pretty bad job of re-checking dependencies, even removing the cache file doesn't work very well.
Just delete and re-extract from the .tar.gz and then cmake again, if you actually have the dependencies then they will be found.
Get sauce from github, read instructions in doc/build-unix.txt. But make can not into compile!
[urs1412#noname bitcoin]$ cd src
[urs1412#noname src]$ make -f makefile.unix
g++ -c -O2 -pthread -Wall -Wextra -Wformat -Wformat-security \
-Wno-unused-parameter -g -DBOOST_SPIRIT_THREADSAFE \
-D_FILE_OFFSET_BITS=64 -I/home/urs1412/w/bitcoin/src \
-I/home/urs1412/w/bitcoin/src/obj -DUSE_UPNP=0 -DUSE_IPV6=1 \
-I/home/urs1412/w/bitcoin/src/leveldb/include \
-I/home/urs1412/w/bitcoin/src/leveldb/helpers \
-DHAVE_BUILD_INFO -fno-stack-protector \
-fstack-protector-all -Wstack-protector \
-D_FORTIFY_SOURCE=2 -MMD -MF obj/alert.d \
-o obj/alert.o alert.cpp \
alert.cpp:6:53: fatal error: boost/algorithm/string/classification.hpp:
No such file or directory
compilation terminated.
make: *** [obj/alert.o] Error 1
td;dr could not build bitcoin, dumping system info
[urs1412#noname src]$ uname -r
3.6.10-4.fc18.x86_64
[urs1412#noname src]$ git log -n 1
commit 77a1e12eed5fc66dce16584696f54988a8c2bf4e
Merge: fe15aa3 0565b71
Author: Gavin Andresen
Date: Wed Apr 24 08:48:06 2013 -0700
Merge pull request #2554 from fanquake/qt-pro-brew-patch
bitcoin-qt.pro Brew patch
I finally was able to build bitcoin-1.8 (not the git sources, although I believe these same steps will be applicable) on my CentOS VPS.
Here are the packages I had to install. Note that I had to build some of these.
As root:
yum install gcc-c++ make
install boost-devel
yum install db4-devel
yum install openssl-devel # but this didn't provide ec.h, hence the next steps
yum install rpm-build
rpm -U ~jcomeau/rpmbuild/RPMS/x86_64/openssl-devel-1.0.0e-1.x86_64.rpm
yum install lynx # for downloading some source packages
yum install python-devel # for building miniupnpc
rpm -i ~jcomeau/rpmbuild/RPMS/x86_64/libminiupnpc9-1.8.20130503-0.1.x86_64.rpm
rpm -i ~jcomeau/rpmbuild/RPMS/x86_64/libminiupnpc-devel-1.8.20130503-0.1.x86_64.rpm
Then as user, make BOOST_LIB_SUFFIX=-mt all test
If you need instructions on building the openssl-devel (the spec file was in the sources and mostly functional) and libminiupnpc-devel (I got the spec file from an OpenSUSE source RPM and adapted it) let me know.
I believe your immediate problem is you didn't install openssl-devel. But you will likely run into these other problems after that, if you don't do some of the steps I did.
Make sure that boost library for gcc is working correctly. Try a test "hello world" program with boost. You can find it in the directory: BOOST_BUILD_PATH/example/hello
Compile it with BOOST_BUILD_PATH/bin/b2 toolset=gcc
If it doesnt work then boost is not correctly installed.
I'm trying to install a package on a machine with no Internet connection. What I want to do is download all the packages and dependences on a machine WITH an Internet connection and then sneaker-net everything to the offline computer.
I've been playing with the apt-get and apt-cache but I haven't figured out a quick and easy way to download the package and dependencies in one swoop to a directory of my choosing. How would I do this? Am I going about this problem correctly?
How would you install offline packages that have a lot of dependencies?
The marked answer has the problem that the available packages on the machine that is doing the downloads might be different from the target machine, and thus the package set might be incomplete.
To avoid this and get all dependencies, use the following:
apt-get download $(apt-rdepends <package>|grep -v "^ ")
Some packages returned from apt-rdepends don't exist with the exact name for apt-get download to download (for example, libc-dev). In those cases, filter out those exact names (be sure to use ^<NAME>$ so that other related names, for example libc-dev-bin, that do exist are not skipped).
apt-get download $(apt-rdepends <package>|grep -v "^ " |grep -v "^libc-dev$")
Once downloaded, you can move the .deb files to a machine without Internet and install them:
sudo dpkg -i *.deb
Same question already answered here:
How to list/download the recursive dependencies of a debian package?
try:
PACKAGES="wget unzip"
apt-get download $(apt-cache depends --recurse --no-recommends --no-suggests \
--no-conflicts --no-breaks --no-replaces --no-enhances \
--no-pre-depends ${PACKAGES} | grep "^\w")
# aptitude clean
# aptitude --download-only install <your_package_here>
# cp /var/cache/apt/archives/*.deb <your_directory_here>
The aptitude --download-only ... approach only works if you have a debian distro with internet connection in your hands.
If you don't, I think it is better to run the following script on the disconnected debian machine:
apt-get --print-uris --yes install <my_package_name> | grep ^\' | cut -d\' -f2 >downloads.list
move the downloads.list file into a connected linux (or non linux) machine, and run:
wget --input-file myurilist
this downloads all your files into the current directory.After that you can copy them on an USB key and install in your disconnected debian machine.
Credits: http://www.tuxradar.com/answers/517
This will download all the Debs to the current directory, and will NOT fail if It can't find a candidate.
Also does NOT require sudo to run sript!
nano getdebs.sh && chmod +x getdebs.sh && ./getdebs.sh
#!/bin/bash
package=ssmtp
apt-cache depends "$package" | grep Depends: >> deb.list
sed -i -e 's/[<>|:]//g' deb.list
sed -i -e 's/Depends//g' deb.list
sed -i -e 's/ //g' deb.list
filename="deb.list"
while read -r line
do
name="$line"
apt-get download "$name"
done < "$filename"
apt-get download "$package"
Note: I used this as my example because I was actually trying to DL the Deps for SSMTP and it failed on debconf-2.0, but this script got me what I need!
Somewhat simplified (and what worked for me) way that worked for me (based on all the above)
Note that dependencies hierarchy can go deeper then one level
Get dependencies of your package
$ apt-cache depends mongodb | grep Depends:
Depends: mongodb-dev
Depends: mongodb-server
Get urls:
sudo apt-get --print-uris --yes -d --reinstall install mongodb-org mongodb-org-server mongodb-org-shell mongodb-org-tools | grep "http://" | awk '{print$1}' | xargs -I'{}' echo {} | tee files.list
wget --input-file files.list
I used apt-cache depends package to get all required packages in any case if the are already installed on system or not.
So it will work always correct.
Because the command apt-cache works different, depending on language, you have to try this command on your system and adapt the command.
apt-cache depends yourpackage
On an englisch system you get:
$ apt-cache depends yourpackage
node
Depends: libax25
Depends: libc6
On an german system you get:
node
Hängt ab von: libax25
Hängt ab von: libc6
The englisch version with the term:
"Depends:"
You have to change the term "yourpackage" to your wish twice in this command, take care of this!
$ sudo apt-get --print-uris --yes -d --reinstall install yourpackage $(apt-cache depends yourpackage | grep " Depends:" | sed 's/ Depends://' | sed ':a;N;$!ba;s/\n//g') | grep ^\' | cut -d\' -f2 >downloads.list
And the german version with the term:
"Hängt ab von:"
You have to change the term "yourpackage" to your wish twice in this command, take care of this!
This text is used twice in this command, if you want to adapt it to your language take care of this!
$ sudo apt-get --print-uris --yes -d --reinstall install yourpackage $(apt-cache depends yourpackage | grep "Hängt ab von:" | sed 's/ Hängt ab von://' | sed ':a;N;$!ba;s/\n//g') | grep ^\' | cut -d\' -f2 >downloads.list
You get the list of links in downloads.list
Check the list, go to your folder and run the list:
$ cd yourpathToYourFolder
$ wget --input-file downloads.list
All your required packages are in:
$ ls yourpathToYourFolder
This will download all packages and dependencies (no already installed) to a directory of your choice:
sudo apt-get install -d -o Dir::Cache=/path-to/directory/apt/cache -o Dir::State::Lists=/path-to/directory/apt/lists packages
Make sure /path-to/directory/apt/cache and /path-to/directory/apt/lists exist.
If you don't set -o Dir::Cache it points to /var/cache/apt,
Dir::State::Lists points to /var/lib/apt/lists (which keeps the index files of available packages)
Both -o options can be used with update and upgrade instead of install.
On different machine run the same command without '-d'
I'm assuming you've got a nice fat USB HD and a good connection to the net. You can use apt-mirror to essentially create your own debian mirror.
http://apt-mirror.sourceforge.net/
On modern Ubuntu systems (for example, 22.04):
apt clean
apt update
apt install --download-only freeipa-client
After you can find deb-files in
ls -l /var/cache/apt/archives/
IF you accept the caveat that there may be dependencies already installed on your system, then the easiest way is to go apt-get install --simulate <your_package>, this will first list all the deps it will install, then copy the list of packages, then apt-get download <the_list_of_packages>
e.g. for qt5-gtk2-platformtheme on a xubuntu-21.04 MINIMAL INSTALL you'll get (after apt-get install --simulate) the following:
libdouble-conversion3 libmd4c0 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5gui5 libqt5network5 libqt5svg5 libqt5widgets5 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-render-util0 libxcb-xinerama0 libxcb-xinput0 libxcb-xkb1 libxkbcommon-x11-0 qt5-gtk-platformtheme qttranslations5-l10n
then you just cd in a folder of your choice, do apt-get download <the_list_above>, and you have them all d/w in there. you can then dpkg -i *.deb
Complementing and automating the exclusion of ALL conflictive dependencies (dependencies not found) by the command given by #onno:
apt-get download $(apt-rdepends <package>|grep -v "^ " |grep -v "^conflictiv-dependency$")
At least for Ubuntu, where the Error Message format is as follows:
E: Can't select candidate version from package <package> as it has no candidate
The following script Downloads all Found Dependencies, Excluding not Found ones:
#!/bin/bash
rm -f error.txt
apt download $(apt-rdepends $1 | grep -v "^ ") 2> error.txt
#IF THERE WAS ERRORS (DEPENDENCIES NOT FOUND)
if [ $(cat error.txt | wc -l) -gt 0 ]
then
partial_command="\("
while read -r line
do
conflictive_package="$(awk '{split($0,array," "); print array[8]}' <<< $line)"
partial_command="$partial_command$conflictive_package\|"
done < error.txt
partial_command="$(awk '{print substr($0, 1, length($0)-2)}' <<< $partial_command)\)"
eval "apt download \$(apt-rdepends $1 | grep -v '^ ' | grep -v '^$partial_command$')"
fi
rm error.txt
It works with me
sudo apt-get reinstall --download-only <your software>
for example
sudo apt-get reinstall --download-only ubuntu-restricted-extras
For accessing installed .deb files, you can look in this path:
/var/cache/apt/archives