See Linux version in terminal

I always, always forget this command. I search it in internet again and again, every time when I need to determine my Linux version. I am going to stop it now – I memorize the command in my notes:

The  lsb_release command provides certain LSB (Linux Standard Base) and
distribution-specific information. The key -a means all possible info, it is not much.

Docker on Ubuntu quickstart

Short tips how to install and use Docker on Ubuntu Server 16.04

Install Docker engine

Check installation

Continue reading

File could only be replicated to 0 nodes instead of minReplication (=1)

Working on newly configured cluster. Can browse HDFS but any write attempt produces an error:

Also I can  create and view tables in Hive, but any insert attempt fails with the same error. Continue reading

Connect by ssh without password

How to configure server and client for SSH authentication without password.

  1. On client generate a new key

    You will be asked to enter passphrase and key file name. I entered empty passphrase and dmitry key name
  2. On client copy generated key to ~/.ssh folder
  3. On client use ssh-copy-id command for adding generated pub key to server’s authorized keys.
  4. On client edit ~/.ssh/config or create if not exists. Add following lines:
  5. On server, if you need to connect as root without password too, edit /etc/ssh/sshd_config, say yes to PermitRootLogin

    Then restart ssh

    Run visudo and edit line, starting from %sudo
  6. Now you can connect server without password

Hive2 metastore configuring

By default, hive runs with embedded derby metastore, which allows only one connection. This article is about how to hive with derby network server. Assume hive is installed to /opt/hive folder

  1. Download derby https://db.apache.org/derby/derby_downloads.html to /opt/derby folder
  2. Start derby server nohup /opt/derby/bin/startNetworkServer &
  3. Edit /opt/hive/conf/hive-site.xml
  4. Start derby
  5. Init metastore
  6. Start hiveserver2 and beeline, should work both simultaneously

    and check in browser http://localhost:10002

     

Hive2 installation

Recently installed Hive 2.1.1 on Ubuntu 16.04. Maybe not optimal steps, but it worked for me.

Installation

1. Download Hive from https://hive.apache.org/downloads.html, unpack archive to folder /opt/hive

2. Copy /opt/hive/conf/hive-default.xml.template to hive-site.xml

Edit hive-site.xml, replace all occurences of ${system:java.io.tmpdir}/${system:user.name} with /tmp/hive Continue reading