Month: August 2016

Aware recruitment. Part 1: Introduction

Daniel, , , , 2 Comments

This article is for you if you work as a recruiter, have to find some Java developers, but have totally no idea what these buzzy words means: JDK, JPA, JSF, EJB, etc.

First of all – the topic is huge. So please note that this article doesn’t cover all answers for all questions and all technology relations. Still – it should cover basics and I hope that you feel much better while talking to developers 🙂 Please remember that I have to simplify a lot of things!

Why is this important?

When anybody starts asking you about technologies or anything tech-related it’s probably easier just answer that it’s not within your competences. But… using this approach you can lose some good candidates. Out there are a lot of similar technologies. Your client can ask to find someone with experience in Hibernate. Hibernate is just an implementation of JPA  so you can check every candidate that have JPA in CV (there are other implementations of JPA). Knowledge about these relations can boost your productivity. Lets be more aware of Java world!

Typical (simple) web application

First thing that you have to be familiar with is basic concept about what kind of developer you are looking for. Look at these terms:

  • Frontend – everything that application users can see,
  • Backend – everything what is “behind” frontend,
  • Fullstack – frontend + backend.

Frontend-Backend

Generally, creating web application, you can see 3 different blocks: frontend (basically this is a web page that user sees with buttons, links and text), backend (this part generates all data for front end and can make some other things – like sending scheduled newsletter, choose some random promotion for every day, check our daily balance and a lot more) and DB (Database – this is a place where we store all persisted data like our clients, products, invoices).

Is that all?

It looks like there is not too much to talk about. BUT in fact there is much more. Developers have their own environment with special tools to build their applications, there are a lot of infrastructure around application and around development environment as well. So if we want to hire developer we will have to check not only technologies that are connected to application. All tools that are used on daily basis by client’s team are also very important. Imagine that you want to hire an accountant and you have a perfect candidate. He have very good knowledge about current law and have experience in similar company to yours. Seems like ideal. Now imagine that this particular person has never seen a computer and any accounting software. In his previous job they do everything in old-school way on paper and he personally has never used computer so far. It could change your mind I think. And that kind of problems occur very often in software industries. Companies use very different set of tools and these tools are not always up to date.

Before we try to go deeper let’s try to describe all kind of “groups” that we will have to cover:

  1. Frontend technologies
  2. Backend technologies
  3. Databases
  4. Operating systems
  5. Infrastructure elements
  6. Building Tools and dependency management
  7. Testing libraries and frameworks
  8. Virtualization solutions
  9. Management methodologies
  10. Management systems
  11. Code Version Control Systems
  12. Continuous Integration and Continuous Delivery Solutions
  13. Other

That’s a lot. And thats why I must split this story to few articles.

 Backend overview

Basically, when we are talking about Java applications we can separate concepts by its “size” or by its purpose (please note that this is how I try to categorize them for You). When we try to categorize concepts by size we will got these 3 elements:

  • Standards – some abstract concepts or standards that developers need to obey while doing particular things. Standards are very important – if particular developer know some library that follow some standards then he or she can quite easily start working with another library that follow same standard. Some examples: JPA, JAXB, JAX-RS, JAX-WS, JMS. Need more? Look here.
  • Frameworks – big things. Imagine that frameworks are some big blocks that cover a lot of problems (In the same time they create a lot of new problems :)). Choosing one particular framework pushes developers to make specific choices within their architecture. Frameworks sometimes covers only one part of application needs, sometimes a lot of them. Examples: Spring, Play, Jersey, CXF, JUnit
  • Libraries – smallest part. You need to know what the most popular libraries are used out there, but remember that a good developer can learn using a new library within a few days. For example: Guava, Lombok, JodaTime.

If we try to categorize them by purpose, we will have something like this (don’t worry if you are not familiar by these names):

  • Endpoint services – these are elements that have to serve “answers”. Endpoint as the name suggests, is a place in the code that will give calculated answers. Imagine that you are just about to buy some books online. This application frontend will have to connect backend and get your current list of books in your basket from particular endpoint. Developer will have to create some “point” that will have to calculate this result. For example: Jersey, Struts.
  • Database services – these elements just simplify developers to work with persisted data (like orders or availability of books in our imaginary store). So every application that use a database also have to use some database services. Examples: Hibernate, MyBatis, Spring-Data
  • Middle services – these elements connects endpoints with database services. Examples: Spring-Security, EJB

Frontend overview

We have few kinds of applications today:

  1. Desktop applications – These are programs that works on your computer directly. For example Word® or Outlook®. Also Browsers are desktop applications 🙂
  2. Web Applications – Works in your browser. For example Amazon®, Gmail® or Facebook®.
  3. Mobile Applications – Apps that works on your mobile phone. There is a lot of web applications that have their own mobile apps and a lot of “only mobile” apps as well.
  4. Console Applications – Apps with very simplified user interface.
  5. No-frontend Applications – Applications that don’t have any frontend.

Desktop applications

There are a lot of technologies that are able to provide particular kind of application. For example Java desktop applications are created mostly in javaFX technology today. A few years ago Swing was the most popular library. There is also AWT– its ancestor. So if you see that someone knows Swing and your client want a developer with JavaFX knowledge you can try to ask if this particular developer have some knowledge about JavaFX or ask your client if Swing knowledge is sufficient. Unfortunately these two don’t have a lot in common. But if you see in clients requirements that he wants Swing and JavaFX you can for sure ask why- because it doesn’t make any sense to write any application in these two at the same time. They are mutually exclusive.

Web applications

Every web page is just:

But, this is a bit more complicated because everything can be generated or delivered in a lot of different ways.

Mobile applications

There are a lot of mobile devices that give possibility to use Java. Almost every device has its own library and approach so it’s quite hard to cover them all. I will try to give a little overview about Android world as it’s most popular platform today.

DB overview

Database is just a container for data. If you don’t want to lose some particular file you store it on your hard drive. In application world database works similar but its far more complicated. There are a lot of different databases:

  • RDBMS – Relational Database Management System. Most popular conception where data can have relations. For example Your basket has one-to-many relation to products. So You have one basket while shopping and you can put as many products as you like in it.
  • NoSQL – Databases that have different conception than tabular idea used in relational ones. If we want to use NoSQL database, we need to choose one as there are a lot of them:

 Summary

The purpose of this intro is to show spectrum of issues that every recruiter should be familiar with in my opinion. I have choosen three most important topics for this article – I hope now its a bit more obvious how this “java world” works. From now on- when you get your client’s expectations try to classify them as suggested above and then try to find similar technologies that can possibly cover same principles or standards. This will help you looking at particular technologies in candidates resumes.

Next chapter will be a bit different we will cover one topic and then I will give you some example requirements from real-world job offer and show how to look at this offer and how used technologies are connected to each other.

Aware recruitment. Part 1: Introduction

CentOS + Graphite + Grafana + NGINX Monitoring

Daniel, , 1 Comment

This article will show you how to create basic environment for monitoring your distributed application in one place using Graphite and Grafana. Recently I did somehow similar configuration on CentOS Linux so I will show you how to make first step into world of monitoring in this particular OS (version 7.2 actually ;))

What we want to achieve?

Basically we want to create environment where any number of applications (hosted via NGINX in this example, but it could be anything) are reporting to single Graphite. Then we want Grafana for creating some nice dashboards that will give us just in time information what’s happening within our applications.

Graphite + Grafana basic environment

 

Of course this could be much more complicated – Nginx can be used only as LoadBalancers for multiple instances for some application modules. Then we want to both – load balancers and apps to tell our graphite about their real load. We will try to cover more complicated solutions in future.

Why Graphite and Grafana?

Basically because it’s somehow standard these days in IT. Grafana is just a platform that will give you nice dashboards.

Of course you can use tool such as Zabbix but this is a bit more different approach. First of all you have to provide more configuration info in zabbix to give you desired charts. Think about this solution that we want to create as a push notification system to Graphite. And everything – no matter how many instances of specific applications you have will show immediately when application start. I’m not familiar if zabbix can show you that many options on one dashboard and adopt to various numbers of data providers. Imagine auto-scalable system where number of instances of particular module can vary.

For Graphite instead of default database (Whisper).

Can I use other linux distro?

Yes you can.

Graphite – installation

Please use all commands with administrative privileges.

Before you start please update your distribution and install EPEL repository:

Now we can install httpd server (Yes Apache – I don’t mind which server will serve grafana and graphite and this is most common solution so you will get more results while searching for any troubleshooting). Also in this step we will install graphite and MariaDB – our MySQL server.

If you don’t want to use MariaDB but different database or just default one you don’t have to install all mysql/mariadb related packages.

Within next steps we will start and enable http server as well as myslq server:

Now we should create database for graphite (please change passwords!):

Graphite – configuration

Lets start to configure our graphite installation 🙂 We will use mostly default configuration so let’s copy default example configuration files:

Now we should configure graphite to use our database. Please edit /etc/graphite-web/local_settings.py  and add our database configuration:

Now we can generate database schema:

Last step is to set some file permissions:

Now we can reconfigure Apache to serve our graphite! Please note that I serve graphite on port 8080.

Last step is to add port 8080 to httpd configuration so Apache will listen on that port. Please add port 8080 so you will have in /etc/httpd/conf/httpd.conf two ports configured:

At the end restart apache and you should have Graphite installed and configured.

Grafana – installation

This one is easy 🙂

And if we want to serve this also via httpd:

Done.

You should now see at http port 80 grafana. Please add graphite as default source. After installation there is only one user: admin with password admin

Security

Also – In my configuration graphite is accessible only from localhost so I don’t add additional security to this part of monitoring environment. All modules have to send data via tcp port 2003 which is accessible internal only. Only one access point for whole monitoring solution is Grafana – it have simple user management so you can use it out of the box. If you need to send data to graphite via internet please read this article before you start.

NGINX Monitoring

Next step is to provide simple but efficient nginx monitoring. First you need to install nginx, collectd and collectd plugin for nginx:

We will use Stub Status Module to give status information from NGINX then we will use collectd to send data over network to installed in earlier steps graphite. On some environments Stub Status Module is already compiled into nginx. If its not please use same steps as in Requirement section on this page.

Now we need to add information about endpoint that will serve our status information. Please add to /etc/nginx/nginx.conf  this code (in server section):

Please note that this endpoint will be accessible only from local machine. If you want to check if its ok. Use wget -q -O - "$@" http://localhost/nginx_status to get information about current nginx status.

Now we have to configure collectd. Everything is in /etc/collectd.conf  file. We want to enable nginx plugin and write_graphite plugin:

Restart nginx:

At this point  you will be able to create and add graphs. In next article in this series I will show you how to configure Grafana to adapt to multiple instances of nginx and show reasonable information just in time.

CentOS + Graphite + Grafana + NGINX Monitoring