Author Archives: Dmitry Pukhov

R vs Python for time series analysis

Working with R and Python simultaneously is a mess. Here is a short comparison how to do things with R and Python to analyze financial data.

Reading the data



Reading financial data



View table



Selecting time series






Train model


.. to be continue


How to check if the port is opened

Case 1. To check whether port 80 is opened on server1 and monitored by some app there:

Case2. If no app listens the port  on server, we need to start some monitoring tool first and then try telnet:

Start listening on server:

Try to connect from client:

HBase import-export utility

Export table to file system

For single machine deployment target directory can be on local file system (use file: instead of hdfs:)


Import into table from file system



Install Anaconda on Ubuntu

Anaconda3 on Ubuntu 18.04



ECDSA host key differs from the key for the IP address

Ssh connection to the server gives the warning below:

Fix: remove entry for this IP

or for specific user

Change hostname on ubuntu 16.04 xenial

All operations from sudo user sudo su

1. Replace old name with new name in files:


2. Run commands in  terminal:

3. Recreate ssh keys on the machine:

After reboot machine should be accessed by new name

Install R forecast package

Attempted to install forecast package in R 3.2 version, on Ubuntu 16.04 LTS Xenial Xerus:

but when installing dependent packages tseries, quadprod, RcppArmadillo, it produced an error:
/usr/bin/ld: cannot find -lblas, cannot find -llalapack

Solution: install these libs in linux terminal:

and  install.packages("forecast") will work in R console

Use ASTSA and ARIMA for time series prediction

Today we will use R astsa (Applied Statistical Time Series Analysis) library to analyze data and predict it with ARIMA model.

Load the data

Look at ACF and PACF

Compare auto correlation and partial autocorrelation functions for diff data

Output: Continue reading

Predict financial data with R

R code template to predict the future with ARIMA auto regression. To try different models,  change the line with fit <- arima(...)

Continue reading

Find jar with class

This helped me to resolve java.lang.ClassNotFoundException.  Bash command scans jar files in the folder and prints who contains given class.