Windows Even Log is PITA, it always has been, It will always be…
Today I had an alert about an unexpected reboot from one of the few Windows Servers I have and I want to find why it happened and who did it.
Everytime I try to search something in the Even Log I want to cry, it’s one the most time consuming and painful activities I can think about…

But today I found a nice Powershell command that will sort this out in no time… or at least the time taken by the system to search inside the damn Event Log, which usually is quite long…

Get-EventLog -LogName System |? {$_.EventID -in (6005,6006,6008,6009,1074,1076)} | ft TimeGenerated,EventId,Message -AutoSize –wrap

I hope this will come handy

As you may heard on march 10th a large fire destroyed part of a big datacenter in Strasbourg owned by OVH (maybe the biggest european service provider), and yes, this blog burned with it.

After the accident there was a huge discussion on the web, flames (sigh…) on Twitter and Reddit about this crazy provider which doesn’t have a disaster recovery plan or some sort of automagic backup, so people get stucked with no options other than start their site/service from scratch…

Some of you may think I’m mad about it and I would run away from this provider… well I’m not and I’ll remain with OVH.

The reasons are very simple, first of all as you can see the blog is back (maybe better than before, things like this always makes you think how can you improve stuff, or at least this is how they work for me) because (surprise surprise!) I had a backup every 6 hours on another location (thanks restic).
The second reason why I decided to stay with OVH is that their vps offer is perfect for my needs, it costs like a shared hosting service and runs so much better, and obviously I can do whatever I want with my private vps, instead of get stucked with only a wordpress hosting service.

And no, I’m not mad with OVH, because even without reading carefully the contract I signed, I knew from the beginning that I had to take care of backups, even if they were included in the service (and they’re not in my case).
Why? Because I want backup made on my way, so I can control them, I can check them, I can figure out the best recovery plan for me.

I understand those who were complaining about backups made in the same location where the burning happened, they payed for a service and it has a flaw (a big one, don’t get me wrong).
But from my perspective there was a bigger flaw, and it was their thinking “ok I paid someone to take care of the backup, job’s done”.
No… no…. NOOOOO!
If you own a service you have the responsibility to take care of the backup, to understand it, to figure out the recovery plan, and to test it; if their backups burned with servers it’s because they missed one, many or all those points.

That’s it, for me the case is closed.

Every now and then people ask me which is my favorite server producer, and every time I honestly don’t know how to reply because they all work pretty well.
What really changes between competitors are technical support and some of the small bits that many people consider irrelevant, but Imho they are very important, one of them, maybe the most important, it the lights-out management interface (LOM).
Every server producer has it’s own LOM interface, but my favorite (and one of the reasons why I prefer Dell servers) is the Dell Drac.

One of the most common problems with Dell Drac is the virtual console which requires Java JRE and obviously this makes people angry because… well basically because people are lazy, most of the time leave the brain turned off and don’t read errors and exceptions…

If you search online “dell drac java error” you’ll find a whole bunch of forums, thread, reddit posts, also useless Chrome extentions for make the damn virtual console work, sometimes those sources are crap, sometimes they contain small bits of the solution, which is changing because there are several versions of Drac devices and obviously they evolved during the years.
These errors always came from the java.security settings, Drac encrypt data transmissions, and old Drac cards use old encryption protocols and cypher suites, so I decided to make a simple patchfile for the java.security file for a quick change and rollback (it’s not a good idea to turn on old unsecure protocols for you JRE).

First of all you have to identify your java.security file, which is inside you JAVA_HOME/lib/security, after that apply this java.security patchfile.

After that open you java settings and add the url of your Drac web interface to the “Security > site exception” list.

That’s all, now you’ll be able to open the vitual console even on an old Drac 5 with the latest JRE (tested right now with JRE 1.8.0_261).

It’s been a while since a wrote something about Eve, and to be honest this is not a State Of The Alliance (SOTA) because… well I’m not an alliance executor and not even a corp CEO, I’m just a regular grunt, as my Twitter bio says I’m just another kender exploring New Eden.

This is not only a post about Eve, in some sense this is a milestone for me because my main character reached the symbolic objective of 100.000.000 skill points; If I sum all the skill points of all my toons I’m reaching about 400.000.000 SP, but you know… the first char is always the most beloved.

Last time I wrote something about Eve I was starting to train one of my biggest and most precious objective, the JF pilot, It was exactly 4 years ago, I reached that goal and despite all the other big objectives (dread and carrier pilots well trainer, two maxed rorqual toons, one almost maxed industrial an reprocessing toon, 6 maxed pi toons, etc etc…) that JF was one of the sweetest and maybe the most precious.

The Italian Alliance I started with died almost 6 months after I joined because of too many elite-pvp players inside, with the core group we joined CO2 back in Tribute, I was part of one of the epic siege of M-OEE8, we fought hard, we lost the first real fought Keepstar citadel and became history.
Then with the entire alliance I moved to Catch and Impass, where we suffered the biggest betrayal in the history of Eve aka “the Judgement Day”.

Thanks to some good friends (and awesome human beings) we moved to Test Alliance Please Ignore in Esoteria, one of the most remote regions of the Eve universe.
We spent almost a year in Test and I have to admit that I loved it, my time in CO2 was great, but the alliance really changed during the years, it became more obsessed on pvp and revenge, almost closed to new players, at some point CO2 became the only alliance without allies and against everyone, and honestly I didn’t like it…
Test was really different, even today I think it’s the most organized alliance I ever seen, awesome wiki, doctrines extremely well documented, very friendly for noobs, everything was great for regular grunts like me, can’t say the same thing for our CEO because Test leadeship seems quite… how can I say… tricky.

After that experience our corp moved into another historic alliance, The Initiative. I always heard about them as smaller but more pvp focused alliance, some kind of elite pvp group inside the Imperium coalition.
At first I was not sure at all and was tempted to leave my corp and stay in Test, but I decided to have faith in our former CEO (I repeat an awsome person) so I jumped into Init.
We started living between Querious (which was the first null region when I started playing Eve) and Fountain, It was tough at first because we lost our habits (we made a lot of huge industrial production in Esoteria) but then I found an equilibrium and things started to work pretty well.

After about six months into the trial alliance Initiative Mercenaries we were promoted as full members of The Initiative, and I have to admit that I was really proud of it, we were parte of one of the most skilled and active groups in the history of Eve.
With Init we made history again, we archived something that everyone in the game considered impossibile, something called the “siege of Rage” which took an entire year of work and preparation and concluded with the destruction of the first Keepstar citadel ever built in the game, we made history, again.

Living in Init is quite different from every other alliance I lived before, in Init I found great fleet commanders, awesome people always helpful and willing to do everything, but it’s a more mature alliance, you must be able to get your stuff, you must be more independent from a logistic point of view, don’t expect the alliance will run for you providing everything you need, you asked to be part of Init, not the opposite.

Now I continue to play, yesterday I lost my first capital into a huge brawl (I used them several times during the years but never lost one) that made almost 1 trillion isk in lost ships, It was awsome.
During the last year I had moments when I really never played, when I thought to leave the game, months spent mainly doing PI, putting skills in the queues and nothing else; it was not an alliance fault, nor my corp fault, it was simply the consequence of huge CCP mistakes, but that’s a different story for a new SOTA rant.

o7

Recently I received some complanings about load problems on an AWS EC2 t2.medium instance with CentOS 7, despite being a development environment it was under heavy load.
I checked logs and monitoring and excluded any kind of attack, after a speech with the dev team it was clear that the load was ok for the applications running (some kind of elasticsearch scheduled bullshit).

The load was 100% from cpu but I noticed some interesting behavior since a couple of weeks with a lot of steal load.

Looking to EC2 CPU Credits it was crystal clear that we ran out of cpu credits, which turned on some heavy throttling.

Since the developers can’t reduce the load from the applications and the management won’t move from EC2, the solution I suggested was to move to a different kind of instance specifically designed for heavy computational workloads and without cpu credits.

So I made some snapshots and launched a new C5 instance, piece of cake, right?
Well no… as soon as I started the new instance it won’t boot, and returned “/dev/centos/root does not exist” on the logs. :\

So what’s going on here?
Pretty simple, there are significant hardware differences between each type of EC2 instance, for example EC2 C type instances have NVMe SSD storage which require a specific kernel module, same for the network interface with ENA module.

The goal here is to make a new init image with these two modules inside, so during the boot the kernel could use these devices, and find a usable volume for boot and nic for network; the only problem is that we can’t simply boot the system using a live distro and build a new init image with those modules already loaded, remember we’re on AWS not on a good old Vmware instance (sigh…).

First of all I terminated the new instance, it was basically useless, and got back to the starting T2 instance.
Check which kernel version you’re using with “uname -a” and build a new init image including nvme and ena modules using mkinitrd, for example:

mkinitrd -v --with=nvme --with=ena -f /boot/initramfs-3.10.0-1062.18.1.el7.x86_64-nvme-ena.img 3.10.0-1062.18.1.el7.x86_64

Using lsinitrd you can check that your new init image has nvme and ena module files inside.

Now you have to edit your grub config file (/boot/grub2/grub.cfg) and change your first menu entry switching from the old init image to the new one.

Save /boot/grub2/grub.cfg file, CHECK AGAIN YOU HAVE A GOOD SNAPSHOT OR AN AMI, and reboot, nothing should have changed.

Now you can make a new snapshot or AMI and build a new instance from it, choose a C type instance and now it should be able to boot properly.

As you can see the new C5 instance have different storage device names, it has a new nic driver (ena) and it has ena and nvme modules loaded.

Life should be easier without the cloud… again.