I have installed CDH4 on CentOS 6.3 64-bit in Pseudo Distributed mode using the following instructions. Everything is set to localhost in the Hadoop configuration files. But, still when I format the name node the below exception appears. When I add an 192.168.1.101 CentOSHost entry to the /etc/hosts file the exception goes away and I am able to run format/start HDFS and run MR jobs.
I want to run MR jobs even when I am not connected to the network without adding an entry to the /etc/hosts file. How to get this done?
12/08/27 22:17:15 WARN net.DNS: Unable to determine address of the host-falling back to "localhost" address
java.net.UnknownHostException: CentOSHost: CentOSHost
at java.net.InetAddress.getLocalHost(InetAddress.java:1360)
at org.apache.hadoop.net.DNS.resolveLocalHostIPAddress(DNS.java:283)
at org.apache.hadoop.net.DNS.(DNS.java:59)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.newBlockPoolID(NNStorage.java:1017)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.newNamespaceInfo(NNStorage.java:565)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:145)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:724)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1095)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1193)
It looks like some where the configuration is returning/ using the hostname as CentOSHost.
What does hostname --fqdn returns to you?
For Hadoop, it is important that name look-up and reverse look-up work successfully. You should be able to resolve the ip-address and resolve hostname from the ip-address (Reverse resolution). This can be tested using the above command.
The entry to /etc/hosts is required for the reverse resolution to work. Unless the entry and the configuration are pointing to localhost. Even in that case the hostname --fqdn should return as localhost.
Related
Trying to set up a network printer with CUPS.
Followed online documentation that stated:
To discover or share printers using DNS-SD/mDNS, setup .local hostname
resolution with Avahi and restart cups.service.
Followed directions for setting up Avahi to the point where avahi-browse --all --ignore-local --resolve --terminate and avahi-resolve-host-name my-domain.local are both working.
But getent hosts my-domain.local fails to resolve. This results in CUPS failing to print because it can't find my-printer.local.
I read the mdns Github page and saw a note that made me think I didn't need a /etc/mdns.allow file.
nss-mdns has a simple configuration file /etc/mdns.allow for enabling
name lookups via mDNS in other domains than .local.
Note: The "minimal" version of nss-mdns does not read /etc/mdns.allow under any circumstances. It behaves as if the file
does not exist.
In the recommended configuration, no /etc/mdns.allow file is present.
But then I saw the last note in that section:
If, during a request, the system-configured unicast DNS (specified in
/etc/resolv.conf) reports an SOA record for the top-level local name,
the request is rejected. Example: host -t SOA local returns something
other than Host local not found: 3(NXDOMAIN). This is the unicast SOA
heuristic.
I tested that out on my machine and sure enough, I was getting something OTHER than Host local not found....
Adding a /etc/mdns.allow file with a line for .local. and for .local and now I can ping my-printer.local.
I'm trying to setup a private ethereum test network using Puppeth (as Péter Szilágyi demoed in Ethereum devcon three 2017). I'm running it on a macbook pro (macOS Sierra).
When I try to setup the ethstat network component I get an "docker configured incorrectly: bash: docker: command not found" error. I have docker running and I can use it fine in the terminal e.g. docker ps.
Here are the steps I took:
What would you like to do? (default = stats)
1. Show network stats
2. Manage existing genesis
3. Track new remote server
4. Deploy network components
> 4
What would you like to deploy? (recommended order)
1. Ethstats - Network monitoring tool
2. Bootnode - Entry point of the network
3. Sealer - Full node minting new blocks
4. Wallet - Browser wallet for quick sends (todo)
5. Faucet - Crypto faucet to give away funds
6. Dashboard - Website listing above web-services
> 1
Which server do you want to interact with?
1. Connect another server
> 1
Please enter remote server's address:
> localhost
DEBUG[11-15|22:46:49] Attempting to establish SSH connection server=localhost
WARN [11-15|22:46:49] Bad SSH key, falling back to passwords path=/Users/xxx/.ssh/id_rsa err="ssh: cannot decode encrypted private keys"
The authenticity of host 'localhost:22 ([::1]:22)' can't be established.
SSH key fingerprint is xxx [MD5]
Are you sure you want to continue connecting (yes/no)? yes
What's the login password for xxx at localhost:22? (won't be echoed)
>
DEBUG[11-15|22:47:11] Verifying if docker is available server=localhost
ERROR[11-15|22:47:11] Server not ready for puppeth err="docker configured incorrectly: bash: docker: command not found\n"
Here are my questions:
Is there any documentation / tutorial describing how to setup this remote server properly. Or just on puppeth in general?
Can I not use localhost as "remote server address"
Any ideas on why the docker command is not found (it is installed and running and I can use it ok in the terminal).
Here is what I did.
For the docker you have to use the docker-compose binary. You can find it here.
Furthermore, you have to be sure that an ssh server is running on your localhost and that keys have been generated.
I didn't find any documentations for puppeth whatsoever.
I think I found the root cause to this problem. The SSH daemon is compiled with a default path. If you ssh to a machine with a specific command (other than a shell), you get that default path. This does not include /usr/local/bin for example, where docker lives in my case.
I found the solution here: https://serverfault.com/a/585075:
edit /etc/ssh/sshd_config and make sure it contains PermitUserEnvironment yes (you need to edit this with sudo)
create a file ~/.ssh/environment with the path that you want, in my case:
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
When you now run ssh localhost env you should see a PATH that matches whatever you put in ~/.ssh/environment.
I had a problem trying to get Apache Geode (v1.0.0-incubating.M2) running on Linux.
The problem was: while I was trying to run gfsh start server --name=server1 example command from the documentation it gave me the following error:
Exception in thread "main" com.gemstone.gemfire.InternalGemFireError: Cannot resolve local host name to an IP address.
It turns out that you need to have your hostname (given by output of hostname command) be present in /etc/hosts file.
In my case, hostname gives an alias as an output (let's say my_alias), so I solved the problem by adding my_ip my_full_domain my_alias line to /etc/hosts.
I have a Cpanel Server.
It send emails correctly expect from 1 domain which hosted on the server , so when I try to send email from that domain using roundcube or Horde I got the errror
SMTP Error (451): Failed to add recipient "recipient#exmple.com" (Temporary local problem - please try later).
does anyone know why and how to fix this?
I found the porblem:
After reviewing the file /var/log/exim_mainlog using
tail -f /var/log/exim_mainlog
I noticed that the error was:
2013-05-29 20:04:28 SMTP connection from [127.0.0.1]:36797 (TCP/IP connection count = 1)
2013-05-29 20:04:28 lowest numbered MX record points to local host: domain.com (while verifying <user#domain.com> from host localhost.localdomain (domain.com) [127.0.0.1]:36797)
2013-05-29 20:04:28 H=localhost.localdomain (domain.com) [127.0.0.1]:36797 sender verify defer for <user#domain.com>: lowest numbered MX record points to local host
2013-05-29 20:04:28 H=localhost.localdomain (domain.com) [127.0.0.1]:36797 F=<user#domain.com> A=dovecot_login:narena temporarily rejected RCPT <recipient#exmple.com>: Could not complete sender verify
2013-05-29 20:04:28 SMTP connection from localhost.localdomain (domain.com) [127.0.0.1]:36797 closed by QUIT
so the main problem was:
lowest numbered MX record points to local host
after couple of search I found the soluation in http://forums.cpanel.net/f5/lowest-numbered-mx-record-points-local-host-73563.html
which was to:
login to WHM and go to Main >> DNS Functions >> Edit MX Entry for the domain
set MX priority to 0 for the related domain and save.
I had the same problem after running a script to fix directory permissions on a cPanel-powered server (CentOS 6.5). I checked the logfile (tail -f /var/log/exim_mainlog) and found this error:
require_files: error for /home/user_name/etc/domain.com: Permission denied
Just ran the following command and the issue was fixed:
chown -R user_name:mail /home/user_name/etc/
Hope this helps someone.
check the the file /var/log/exim_mainlog to see more information about the error
tail -f /var/log/exim_mainlog
while trying to send email
Check your MX Entry in Cpanel, if the existing domain priority is less than or equals to 0, set it to 1. Mine is fixed. Hope it will help you.
Wow, after about an hour of searching and meddling with different files, I'd caution any novice not to venture out editing anything before you have a backup or image if your server, as you can cause irrevocable damage to your server. So many people talking garbage about what you should do or test without any real solution.
Anyways, here's what worked for me:
Real problem: Exim was updated to latest version which has loads of bugs like this issue.
How I fixed my server:
Authenticate to Linux via SSH and run the command lines through which we download and install the old version of EXIM.
Command Line 1: wget https://ca1.dynanode.net/exim-4.93-3.el7.x86_64.rpm
Command Line 2: rpm -Uvh --oldpackage exim-4.93-3.el7.x86_64.rpm
Command Line 3: systemctl restart exim
Command Line 4: Systemctl restart clamd
Command Line 5: systemctl restart spamassassin
Optional: just type "Reboot" to restart your server
The command lines above does the following:
Downloads the old package (I'm sure you can google other sources with this file)
Install the old package without prompt
Restart the Exim service
Restart the Clamd Service (AV)
Restart the spamassassin service (Spam Filter)
Restart outlook or whatever you use for mail client and send an email. Mine works, hope yours do too.
I need to setup a hadoop/hdfs cluster with one namenode and two datanodes. I am aware of conf/slaves file which lists the machines datanodes are running. But how can I specify where hadoop/hdfs is locally installed on slave node? Also the user account to start hdfs there?
Edit: in log files, I find following error, when I tried to start-dfs.sh
ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: file:///
The user is expected to be the same as on the master node. The location of the actual data can be modified by changing the dfs.data.dir node inhadoop-site.xml.