Maybe I’m the last user on earth using Cygwin (I doubt it considering the project is well maintained and its newsletter is very active) but lately I noticed that opening the terminal window takes really long time.

After digging a little bit I found the problem came from my working pc logged on an Active Directory domain (for instance I’m not an AD lover… but that’s another story) and obviously Stackoverflow had the solution (I’m quite surprise we didn’t found the cure for cancer or the meaning of life on it… yet).

https://superuser.com/questions/877051/cygwin-from-windows-is-very-slow-laggy/1183283

I report it here just in case some day Stackoverflow will be bought by some lunatic billionaire that will decide to turn it into ashes…

Rough solution:

$ mkpasswd -l -c >/etc/passwd
$ mkgroup -l -c >/etc/group
$ echo 'passwd: files' >> /etc/nsswitch.conf
$ echo 'group: files' >>/etc/nsswitch.conf

More elegant solution

$ cygserver-config
Generating /etc/cygserver.conf file

Warning: The following function requires administrator privileges!

Do you want to install cygserver as service?
(Say "no" if it's already installed as service) (yes/no) yes

The service has been installed under LocalSystem account.
To start it, call `net start cygserver' or `cygrunsrv -S cygserver'.

Further configuration options are available by editing the configuration
file /etc/cygserver.conf. Please read the inline information in that
file carefully. The best option for the start is to just leave it alone.

Basic Cygserver configuration finished. Have fun!

Windows Even Log is PITA, it always has been, It will always be…
Today I had an alert about an unexpected reboot from one of the few Windows Servers I have and I want to find why it happened and who did it.
Everytime I try to search something in the Even Log I want to cry, it’s one the most time consuming and painful activities I can think about…

But today I found a nice Powershell command that will sort this out in no time… or at least the time taken by the system to search inside the damn Event Log, which usually is quite long…

Get-EventLog -LogName System |? {$_.EventID -in (6005,6006,6008,6009,1074,1076)} | ft TimeGenerated,EventId,Message -AutoSize –wrap

I hope this will come handy

As you may heard on march 10th a large fire destroyed part of a big datacenter in Strasbourg owned by OVH (maybe the biggest european service provider), and yes, this blog burned with it.

After the accident there was a huge discussion on the web, flames (sigh…) on Twitter and Reddit about this crazy provider which doesn’t have a disaster recovery plan or some sort of automagic backup, so people get stucked with no options other than start their site/service from scratch…

Some of you may think I’m mad about it and I would run away from this provider… well I’m not and I’ll remain with OVH.

The reasons are very simple, first of all as you can see the blog is back (maybe better than before, things like this always makes you think how can you improve stuff, or at least this is how they work for me) because (surprise surprise!) I had a backup every 6 hours on another location (thanks restic).
The second reason why I decided to stay with OVH is that their vps offer is perfect for my needs, it costs like a shared hosting service and runs so much better, and obviously I can do whatever I want with my private vps, instead of get stucked with only a wordpress hosting service.

And no, I’m not mad with OVH, because even without reading carefully the contract I signed, I knew from the beginning that I had to take care of backups, even if they were included in the service (and they’re not in my case).
Why? Because I want backup made on my way, so I can control them, I can check them, I can figure out the best recovery plan for me.

I understand those who were complaining about backups made in the same location where the burning happened, they payed for a service and it has a flaw (a big one, don’t get me wrong).
But from my perspective there was a bigger flaw, and it was their thinking “ok I paid someone to take care of the backup, job’s done”.
No… no…. NOOOOO!
If you own a service you have the responsibility to take care of the backup, to understand it, to figure out the recovery plan, and to test it; if their backups burned with servers it’s because they missed one, many or all those points.

That’s it, for me the case is closed.

Every now and then people ask me which is my favorite server producer, and every time I honestly don’t know how to reply because they all work pretty well.
What really changes between competitors are technical support and some of the small bits that many people consider irrelevant, but Imho they are very important, one of them, maybe the most important, it the lights-out management interface (LOM).
Every server producer has it’s own LOM interface, but my favorite (and one of the reasons why I prefer Dell servers) is the Dell Drac.

One of the most common problems with Dell Drac is the virtual console which requires Java JRE and obviously this makes people angry because… well basically because people are lazy, most of the time leave the brain turned off and don’t read errors and exceptions…

If you search online “dell drac java error” you’ll find a whole bunch of forums, thread, reddit posts, also useless Chrome extentions for make the damn virtual console work, sometimes those sources are crap, sometimes they contain small bits of the solution, which is changing because there are several versions of Drac devices and obviously they evolved during the years.
These errors always came from the java.security settings, Drac encrypt data transmissions, and old Drac cards use old encryption protocols and cypher suites, so I decided to make a simple patchfile for the java.security file for a quick change and rollback (it’s not a good idea to turn on old unsecure protocols for you JRE).

First of all you have to identify your java.security file, which is inside you JAVA_HOME/lib/security, after that apply this java.security patchfile.

After that open you java settings and add the url of your Drac web interface to the “Security > site exception” list.

That’s all, now you’ll be able to open the vitual console even on an old Drac 5 with the latest JRE (tested right now with JRE 1.8.0_261).

Recently I received some complanings about load problems on an AWS EC2 t2.medium instance with CentOS 7, despite being a development environment it was under heavy load.
I checked logs and monitoring and excluded any kind of attack, after a speech with the dev team it was clear that the load was ok for the applications running (some kind of elasticsearch scheduled bullshit).

The load was 100% from cpu but I noticed some interesting behavior since a couple of weeks with a lot of steal load.

Looking to EC2 CPU Credits it was crystal clear that we ran out of cpu credits, which turned on some heavy throttling.

Since the developers can’t reduce the load from the applications and the management won’t move from EC2, the solution I suggested was to move to a different kind of instance specifically designed for heavy computational workloads and without cpu credits.

So I made some snapshots and launched a new C5 instance, piece of cake, right?
Well no… as soon as I started the new instance it won’t boot, and returned “/dev/centos/root does not exist” on the logs. :\

So what’s going on here?
Pretty simple, there are significant hardware differences between each type of EC2 instance, for example EC2 C type instances have NVMe SSD storage which require a specific kernel module, same for the network interface with ENA module.

The goal here is to make a new init image with these two modules inside, so during the boot the kernel could use these devices, and find a usable volume for boot and nic for network; the only problem is that we can’t simply boot the system using a live distro and build a new init image with those modules already loaded, remember we’re on AWS not on a good old Vmware instance (sigh…).

First of all I terminated the new instance, it was basically useless, and got back to the starting T2 instance.
Check which kernel version you’re using with “uname -a” and build a new init image including nvme and ena modules using mkinitrd, for example:

mkinitrd -v --with=nvme --with=ena -f /boot/initramfs-3.10.0-1062.18.1.el7.x86_64-nvme-ena.img 3.10.0-1062.18.1.el7.x86_64

Using lsinitrd you can check that your new init image has nvme and ena module files inside.

Now you have to edit your grub config file (/boot/grub2/grub.cfg) and change your first menu entry switching from the old init image to the new one.

Save /boot/grub2/grub.cfg file, CHECK AGAIN YOU HAVE A GOOD SNAPSHOT OR AN AMI, and reboot, nothing should have changed.

Now you can make a new snapshot or AMI and build a new instance from it, choose a C type instance and now it should be able to boot properly.

As you can see the new C5 instance have different storage device names, it has a new nic driver (ena) and it has ena and nvme modules loaded.

Life should be easier without the cloud… again.