Links generated by autotools should be replaced before uploading? - open-source

My main question is that the autotools created links to INSTALL, COPYING, missing, install-sh, and depcomp. When I tried to view them I saw that they were uploaded as links so I replaced them with real files so they are viewable. Am I missing something fundamental? When I unpack my gz file from 'make dist' this is what it looks like:
Distribution Tree (Minus Source Directories)
-rw-r--r-- 1 ojblass users 18591 2009-05-30 03:23 Makefile.in
-rwxr-xr-x 1 ojblass users 136168 2009-05-30 03:20 configure
drwxr-xr-x 3 ojblass users 4096 2009-05-30 03:20 autom4te.cache
-rw-r--r-- 1 ojblass users 32230 2009-05-30 03:20 aclocal.m4
-rw-r--r-- 1 ojblass users 251 2009-05-30 03:20 configure.ac
-rw-r--r-- 1 ojblass users 626 2009-05-30 03:11 AUTHORS
-rwxr-xr-x 1 ojblass users 120 2009-05-30 03:11 autogen.sh
-rw-r--r-- 1 ojblass users 737 2009-05-30 03:11 ChangeLog
-rw-r--r-- 1 ojblass users 35147 2009-05-30 03:11 COPYING
-rwxr-xr-x 1 ojblass users 17867 2009-05-30 03:11 depcomp
-rwxr-xr-x 1 ojblass users 199 2009-05-30 03:11 example.pl
-rwxr-xr-x 1 ojblass users 152 2009-05-30 03:11 example.sh
-rw-r--r-- 1 ojblass users 9512 2009-05-30 03:11 INSTALL
-rwxr-xr-x 1 ojblass users 13620 2009-05-30 03:11 install-sh
-rw-r--r-- 1 ojblass users 215 2009-05-30 03:11 Makefile.am
-rwxr-xr-x 1 ojblass users 11135 2009-05-30 03:11 missing
-rw-r--r-- 1 ojblass users 75 2009-05-30 03:11 NEWS
-rwxr-xr-x 1 ojblass users 507 2009-05-30 03:11 profile.sh
-rw-r--r-- 1 ojblass users 2605 2009-05-30 03:11 README
-rw-r--r-- 1 ojblass users 201 2009-05-30 03:11 README_developers
-rwxr-xr-x 1 ojblass users 382 2009-05-30 03:11 run.sh
-rw-r--r-- 1 ojblass users 481 2009-05-30 03:11 TODO
-rwxr-xr-x 1 ojblass users 117 2009-05-30 03:11 usefull.sh
I am planning on removing the README_developers and making two sections in the README. I am also looking at removing the run.sh and profile.sh and making them part of a make test target (some reading required). I do not think that a TODO item belongs in the source distribution but maybe it is okay to have it in the source tree of the project. Any additional pointers above and beyond the links question is appreciated.

Short answer: Leaving those symlinks (other than INSTALL) would be
faked conformance to GNU coding standards.
By default automake perform check for conformance to GNU standards
(requires following files to exist: INSTALL, NEWS, README,
COPYING, AUTHORS and ChangeLog). One may turn of that check (and
safely remove some of those files) by passing --foreign option to
automake (to do it edit autogen.sh and rerun it).
The links were created when automake was invoked with --add-missing
options which unless --copy option was given creates symlinks for
missing files rather then copying them. This is in order to keep those
files (actually INSTALL only) up to date whenever you install newer
automake. Regerding Jonathan's answer those symlinks are not an issue: all
distributed files are copied to separate directory before making tarball.
Change them to regular files if you want to edit them.
Other files (README_developers, run.sh, profile.sh, TODO
etc.) were probably generated by IDE you are using and added to
EXTRA_DIST variable in top Makefile.am. You may remove them from
distribution by editing EXTRA_DIST and afterward you may also remove
them from source.
The rest is automatically generated by autoconf and automake:
aclocal.m4
autom4te.cache
configure
depcomp
install-sh
Makefile.in
missing
If you want to further unclutter your top source directory you may add
AC_CONFIG_AUX_DIR([scripts]) to configure.ac. This way some of the scripts
will find place in scripts directory.
Update:
The GNU coding standards merely describe the requirement of those
documentation files to be present within distribution and what
information should be included there. The --add-missing option is to
remind programmer what files should be written. Obviously having empty
NEWS or AUTHORS file won't make the project more conformant to standard.
Only ChangeLog file have rigid requirements on it's format.
In some projects ChangeLog is automatically generated from properly
formated commit messages. On Darcs it's simply darcs changes >ChangeLog.
If you use Subversion you may look at: svn2log, svn2cl.
As mentioned INSTALL file might be reasonable to keep as symlink but only
if there is no project specific information needed about the installation
(i.e. no extra configure script arguments etc.).

The TODO file can go in the distribution; it is an indication to consumers of things that you think are deficient in the product, and they are a pointer to potential contributors to areas where they could perhaps help improve the product. (Also, if you fell under a bus, it would help other people take over using the current release material.)
It might be worth using a sub-directory to hold the bulk of configuration material.
Regarding your main question - were the files 'links' or 'symlinks'? Symbolic links are less useful for packaging unless you tell (GNU) tar to follow them anyway ('-h' or '--dereference'). If the only symlinks are to those files, that works; if you use symbolic links in the distribution itself, it may be more counter-productive.

Related

How to uninstall mySQL when "rm /usr/local/mysql" returns permissions error?

I have been having problems logging into MySQL through phpMyAdmin so I decided to uninstall both on my computer. To my surprise, I do not even have the permissions to uninstall MySQL.
I tried to uninstall MySQL using the command "sudo rm /usr/local/mysql" and the terminal returned that I did not have permission.
I looked up a StackOverflow question about not having access to /usr/local and a reply asked the user to do sudo $chown, which my terminal said I did not have permission to do.
I did have permission to do ls -la /usr/local, and this is what the terminal returned:
total 0
drwxr-xr-x 16 root wheel 512 Jun 6 22:32 .
drwxr-xr-x# 9 root wheel 288 Mar 27 2018 ..
-rw-r--r-- 1 x wheel 0 Feb 19 23:21
.com.apple.installer.keep
drwxrwxr-x 4 x admin 128 Jun 7 2018 Cellar
drwxrwxr-x 17 x admin 544 May 22 2018 Homebrew
drwxrwxr-x 61 x admin 1952 Jun 7 2018 bin
drwxrwxr-x 3 x admin 96 May 22 2018 etc
drwxr-xr-x 10 x wheel 320 May 22 2018 git
drwxrwxr-x 3 x admin 96 May 22 2018 lib
drwxrwxr-x 5 x wheel 160 May 22 2018 libexec
lrwxr-xr-x 1 x wheel 30 Jun 6 22:32 mysql -> mysql-5.7.26-
macos10.14-x86_64
drwxr-xr-x 13 x wheel 416 Jun 6 22:32 mysql-5.7.26-macos10.14-
x86_64
drwxrwxr-x 4 x admin 128 Jun 7 2018 opt
drwxr-xr-x 3 x wheel 96 May 22 2018 remotedesktop
drwxrwxr-x 5 x admin 160 May 22 2018 share
drwxrwxr-x 3 x admin 96 May 22 2018 var
I am shocked about the "remotedesktop" line, but I hope it is innocent considering it shows up the same day as Homebrew. Please help me understand these results and what to do next.
You can't remove a directory with rm unless you use the recursive flag (-r). You should get an "is a directory" error, not a permissions error. You'll probably want to include the "force" flag (-f) to avoid having to confirm each deletion.
That's also a symlink so you need to remove the specific instance of it, or adapt your command to remove anything MySQL-ish using a wildcard:
rm -rf /usr/local/mysql*
As always, pay extremely close attention to what you're doing when using recursive deletes and sudo. A single space in the wrong place can utterly ruin your day. Triple check before executing these commands.

DC/OS on GCE Ubuntu

Is there any link/documentation available around installing DC/OS on Google Compute Engine where instances are ubuntu 16.04 instances including the bootstrap node instead of CentOS 7?
Currently , the documents I find use Ansible and CentOS 7 on GCE as below.
https://dcos.io/docs/1.7/administration/installing/cloud/gce/
Short answer: Debian based distributions are currently (at least up to DC/OS 1.10) not supported.
Long answer: It's possible, but requires some extra steps.
DC/OS doesn't use any RedHat specific features. Most important differences could be solved by symlinks for few system binaries, as RedHat systems have different paths and systemd doesn't support $PATH variable in service definition. You'll need following:
sudo apt-get install libcurl3-nss ipset selinux-utils curl unzip bc
sudo ln -s /bin/mkdir /usr/bin/mkdir
sudo ln -s /bin/ln /usr/bin/ln
sudo ln -s /bin/tar /usr/bin/tar
sudo ln -s /bin/rm /usr/bin/rm
sudo ln -s /usr/sbin/useradd /usr/bin/useradd
sudo ln -s /bin/bash /usr/bin/bash
sudo ln -s /sbin/ipset /usr/sbin/ipset
Another requirements are:
systemd with version >=200
Docker >=1.6
Slightly outdated scripts from John Omernik, there's also puppet module (I'm the author). For further details see discussion on DC/OS Jira.
Next step is manual DC/OS compilation (it might sound scary, but actually it's very easy). C++ components (especially mesos-slave) are dependent on system libraries and they'd better be linked to proper libraries.
apt install python3-venv build-essential git
git clone https://github.com/dcos/dcos
./build_local.sh
Resulting "image" will be located in:
$HOME/dcos-artifacts/testing/`whoami`/dcos_generate_config.sh
You can copy it to your bootstrap server and extract:
bash dcos_generate_config.sh --genconf
after updating genconf/config.yaml you can start a container for serving the installation scripts:
docker run -d -p 9090:80 -v $PWD/genconf/serve:/usr/share/nginx/html:ro nginx
On a new node simply fetch the installation script:
rm -rf /tmp/dcos && mkdir /tmp/dcos && cd /tmp/dcos && curl -O http://bootstrap.example.com:9090/dcos_install.sh
bash dcos_install.sh slave
Unless you don't want to run packages from DC/OS Universe (like Elastic, Kafka, etc.) that depends on libmesos-bundle, you might be just fine. The bundle is fetched into each executor's directory, it includes numerous libraries, such as libmesos.so
...
-rwxr-xr-x 1 nobody nogroup 55077256 Jun 28 19:50 libmesos-1.4.0.so
-rwxr-xr-x 1 nobody nogroup 1487 Jun 28 19:50 libmesos.la
lrwxrwxrwx 1 nobody nogroup 17 Jun 28 19:50 libmesos.so -> libmesos-1.4.0.so
-rwxr-xr-x 1 nobody nogroup 398264 Jun 28 19:53 libpcre.so.1
-rwxr-xr-x 1 nobody nogroup 121296 Jun 28 19:53 libsasl2.so.3
-rwxr-xr-x 1 nobody nogroup 155744 Jun 28 19:53 libselinux.so.1
-rwxr-xr-x 1 nobody nogroup 454008 Jun 28 19:53 libssl.so.10
-rwxr-xr-x 1 nobody nogroup 999944 Jun 28 19:53 libstdc++.so.6
-rwxr-xr-x 1 nobody nogroup 79000 Jun 28 19:53 libsvn_delta-1.so.0
-rwxr-xr-x 1 nobody nogroup 1820208 Jun 28 19:53 libsvn_subr-1.so.0
-rwxr-xr-x 1 nobody nogroup 20040 Jun 28 19:53 libuuid.so.1
-rwxr-xr-x 1 nobody nogroup 90664 Jun 28 19:53 libz.so.1
drwxr-xr-x 3 nobody nogroup 4096 Jun 28 19:53 mesos
drwxr-xr-x 2 nobody nogroup 4096 Jun 28 19:37 pkgconfig
Some libraries might be compatible with your system, but the versions between CentOS and Debian might (and will) differ. You might encounter errors like:
libmesos-bundle/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by curl)
which will cause that all agent based health checks that use curl won't work, therefore most instances will refuse to start.

Apache doesn't allow to load other files than index.html

Before you ask, I tried to search for similar questions, none of them had precisely my problem.
I'm new at this, and I installed Apache2, PHP5, and MySQL5 on a RasPi running Raspbian.
As needed, I put the index.html in /var/www/html and when I type my Pi's IP alone or followed by /index.html, index.html runs its code properly and all elements appear.
Problem: In addition I put my JS and CSS files in that same folder, as well as a few photos to display.
The browser reads the HTML thoroughly, but doesn't load the thereby linked JS and CSS files nor the images. Putting the files in subfolders also didn't help.
When I type in the IP followed by a /style.css (name of my linked CSS file), I get a 403 Forbidden. The errors log also shows this:
file permissions deny server access: /var/www/html/photo.jpg, referer: http://192.168.178.120/
access to /images/dark.jpg denied because search permissions are missing on a component of the path, referer: http://192.168.178.120/
I tried researching this but nothing helped. When my JS and CSS are inside my HTML, everything works fine except the images
Concluded: Apache doesn't allow the browser to load files other than index.html.
Do you know what the issue might be? Is there a specific folder where I should put my other files? Thanks in advance.
Based on the result from ls, your files are not accessible to Apache.
Short answer:
Run the following two commands:
find /var/www/html -type f -exec chmod 644 {} \;
find /var/www/html -type d -exec chmod 755 {} \;
Longer explanation:
pi#raspberrypi:~ $ ls -l /var/www/html
total 2976
-rw------- 1 pi pi 60146 Jul 23 22:11 bnw.jpg
-rw------- 1 pi pi 202851 Jul 23 22:11 color.jpg
-rw------- 1 pi pi 617185 Jul 23 21:27 dark.jpg
-rw------- 1 pi pi 2028727 Jul 23 22:11 effect.jpg
drwx------ 2 pi pi 4096 Jul 23 21:33 images
-rw------- 1 pi pi 2238 Jul 23 22:11 index.css
-rw-r--r-- 1 pi root 1261 Jul 23 22:11 index.html
-rw------- 1 pi pi 108397 Jul 23 22:11 photo.jpg
-rw------- 1 pi pi 538 Jul 23 22:11 script.js
-rw------- 1 pi pi 2238 Jul 23 21:33 style.css
The symbols at the beginning of the line have the following meaning:
d: indicates a directory
r: means the file/directory is readable
w: means the file/directory is writable
x: means the file/directory is executable
-: means none of the above apply
These symbols are grouped as follows:
drwxrwxrwx
^^^ apply to all users on the system
^^^--- apply to all users in the group that owns the file
^^^------ apply to the user that owns the file
Where it says pi pi it's referring to the file's owner: the user pi in the group pi. Your index.html file is an exception, because it's owned by the group root.
That's not really important for now, other than that you need to realise that Apache usually runs as the user www-data in the group www-data. This varies a bit from system to system, but that's the most common. What that means is that for Apache to be able to access a file, it must be made available to either the www-data user or the www-data group (or both).
In your case, the files are owned by the user pi and the group pi (with the exception of index.html, which is owned by the group root. Since Apache is not that user and is most likely not in either of those groups, that means the file permissions for "all users on the system" must be set correctly for Apache to be able to access the file.
As you can see, index.html is set to be readable for all users on the system:
-rw-r--r-- 1 pi root 1261 Jul 23 22:11 index.html
^^^ all users on the system may read from this file, but may not write and may not execute the file
^^^--- all users in the group "root" may read from this file, but may not write and may not execute the file
^^^------ the user "pi" may read from and write to this file, but may not execute the file
All other files, including the images directory, are only accessible to the "pi" user:
-rw------- 1 pi pi 2028727 Jul 23 22:11 effect.jpg
^^^ all users on the system may not read from, write to or execute this file
^^^--- all users in the group "pi" may not read from, write to or execute this file
^^^------ the user "pi" may read from and write to this file, but may not execute it
So for effect.jpg to be made available to Apache, you need to change the file permissions to this:
-rw----r-- 1 pi pi 2028727 Jul 23 22:11 effect.jpg
^^^ all users on the system (including Apache) may read from this file
To do that, you use the chmod command. There are two ways to flip that permission:
chmod o+r effect.jpg
chmod 604 effect.jpg
chmod o+r means "add 'r' permission to the 'other users' category" (you'd use u+r or g+r to change the user or group permissions).
chmod 604 means "set permissions for the user to 6, group to 0 and others to 4" - where the numbers are a binary sum of the permissions: 1 (executable), 2 (writable), 4 (readable).
Directories need a little more work:
drwx------ 2 pi pi 4096 Jul 23 21:33 images
To allow the file system to actually open a directory and read the files within, it needs to be executable for the user that's trying to access its content. So to allow Apache to read any file from this folder, it will need to have the following permissions:
drwx---r-x 2 pi pi 4096 Jul 23 21:33 images
^^^ all users on the system may read from this directory and execute it
To do this, use the same principle:
chmod o+rx images (other users, add readable and executable permissions)
chmod 705 images (set read(4)+write(2)+execute(1) for owner and read(4)+execute(1) for all other users)
Now, although it's not strictly necessary if the owner and group are the same, it is best practice to make sure that the group has the same permissions as the "all other users" category. So instead of giving files 604 (-rw----r--) and folders 705 (drwx---r-x), it's best practice to give them 644 (-rw-r--r--) and 755 (drwxr-xr-x). If you're working in an environment where multiple developers need to be able to modify the files, they should be in the same user group and best practice is to give the group the same permissions as the owner, so 644 (-rw-rw-r--) and 775 (drwxrwxr-x).
Finally, you don't want to have to change all file permissions manually. This particular project seems to be relatively small, but it's still annoying work. Fortunately, we can use the find command to perform a batch update.
find will list the full contents of the given folder, including subfolders, which you can then filter or perform actions on.
find /var/www/html -type f
This will list all entries within /var/www/html or any subfolders that are a regular file.
find /var/www/html -type d
This will list all entries within /var/www/html or any subfolders that are directories.
We can use -exec to tell find to automatically perform a certain command on each file/folder it finds:
find /var/www/html -type f -exec chmod 644 {} \;
The {} is a placeholder where find will put each filename. The \; is needed to inform find that no further arguments for the -exec command will be provided, so we can optionally add other arguments for find itself.
So the above command will fix the permissions for all files, the following command will fix the permissions for all folders:
find /var/www/html -type d -exec chmod 755 {} \;
After that, Apache should have access to all files and folders in /var/www/html. Keep in mind that, every time you create a new file in that folder, you need to check the permissions and fix it if necessary.

File owner changed after editing it

[root#MGWSDT_FEWS ~]# ll file
-rw-r--r-- 1 root bill 0 Aug 14 17:28 file
[root#MGWSDT_FEWS ~]# su - bill
$ vi /root/file
I edited this file and wq!
Now bill becomes the file owner:
$ ll /root/file
-rw-r--r-- 1 bill bill 16 Aug 14 17:29 /root/file
Why? So strange!
bill cant edit the file. hess part of the group which only has read access.
So switching to bill you would expect a permissions error when you try and write.
In this case bill is also the directory owner, so whats actually happening is the file is being removed, and recreated now with bill as the owner.
:w !sudo tee %
would write as root and keep the permissions

Cannot clone repository ("Operation not permitted: <workspace_folder>/<project_name>/.hg/requires")

Sorry in advance for my bad english...
I'm trying to clone an hg repository using Eclipse on Ubuntu.
I always have the following error message which is exactly the same when I perform a "hg clone" command :
"Operation not permitted: <workspace_folder>/<project_name>/.hg/requires"
Here is the /.hg content :
ls -al .hg/
total 40K
drwxrwxr-x 4 www-data web 4.0K Apr 18 09:33 .
drwxrwxr-x 3 www-data svn 4.0K Mar 5 17:52 ..
-rwxrwxr-x 1 www-data web 57 Mar 5 17:48 00changelog.i
drwxrwxr-x 2 www-data web 4.0K Apr 18 09:33 cache
-rwxrwxr-x 1 www-data web 40 Mar 5 17:51 dirstate
-rwxrwxrwx 1 www-data web 40 Apr 18 21:45 requires
drwxrwxr-x 3 www-data web 4.0K Apr 18 09:33 store
-rw-rw-r-- 1 nico web 0 Apr 18 09:33 undo.bookmarks
-rw-rw-r-- 1 nico web 7 Apr 18 09:33 undo.branch
-rw-rw-r-- 1 nico web 38 Apr 18 09:33 undo.desc
-rw-rw-r-- 1 nico web 40 Apr 18 09:33 undo.dirstate
And here is the /.hg/requires file content :
revlogv1
store
fncache
dotencode
Here is the output of the hg clone command :
running ssh nico#www.there.com "hg -R /var/dev/projects/my_hg_project serve --stdio"
sending hello command
sending between command
nico#www.there.com's password:
remote: 145
remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch stream unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
remote: 1
destination directory: my_hg_project
abandon : Operation not permitted : /media/data/workspaces/my_workspace/my_hg_project/.hg/requires
I tried many things such as chmod and chown... I'm not a linux expert so I googled my error message but there is not many results.
Has anyone an idea about this?
Thank you very much in advance
You're right to be thinking file permissions because that's the root cause, but I'm a little confused. Are you showing up the file permissions on the server, but seeing that message on the your workstation? Where exactly are you trying to clone from and to? Are you cloning over http or ssh? What user are you running the clone command as?