Script

Yet another ESXi pxe boot solution

Yet another pxe boot solution

Hopefully this isn’t just “yet another pxe boot solution” – It may not be as smart as puppet or chef, but hopefully you come to see that this is at least semi automated and a bit clever. So what sets this apart from so many other pxe solution/kickstart.

Well first of it a custom build if you will, meaning its tailored to the environment, in this case it means HP C7000 enclosures and network documentation in excel spreadsheet. So if you run HPs C7000 enclosure and your network team just loves excel spreadsheet, this might be a solution for you aswell.

 

The server

This is installed on a VM in the management cluster which services the VMware infrastructure, this way I feel comfortable having this as a VM. If you want to you could do it as a physical server, but why do that if your infrastructure is properly protected. The OS it self is Ubuntu 12.04 LTS, which was the flavor available at the time the PXE solution was created. On top of that the following services were installed.

 

Component

Version

Syslinux

3.86

tftpd-hpa

5.2-1ubuntu

isc-dhcp-server (DHCPD3)

4.1.ESV-R4

apache2

2.2.22

Again version wise this was what was available at the time the PXE server was created. (There is no reason to not use the latest version.)

That’s the story of the software used, hardware wise, I’ve talk a bit about, its a VM with the following specs.

 

Attribute

Specification

HW version

7

Memory

2GB

vCPU

1

Hard disk 1 (sda)

16 GB

Hard disk 2 (sdb)

40 GB

Network

The network for the ESXi, hosts VMkernel port

There is two hard disks, the first is used for OS, logs etc. and the second one is used for ISO etc. The second harddisk is mount under it’s own directory in the root of the partition, this needs to be sized after the amount of data you are expected to use.

Paths

Hard disk 2 is mounted under /media and all files are mounted under /media/pxeserver

ESXi image: /media/pxeserver/tftpboot/esxi/

Kickstart files: /media/pxeserver/www/kickstart/

PXE menu files: /media/pxeserver/tftpboot/pxelinux.cfg/

Not Yet another ESXi pxe boot solution

As I state in the beginning this is very much a custom solution fitted for the needs of this organization. To understand why it came to be this way, I’m going talk about some of the challenges this solution were set out to fix (and maybe in another blog post expand on what it expanded to fix of other challenges). Before I do that I’ll like to set the stage – Before this solution came to be, installations were done manually following a word doc, which was not in a strict step by step manner and also used a few different technologies, perl, powershell, esxcli etc. This made the procedure complicated, slow, error prone and susceptible to human error. It was these challenges I set out to fix doing this custom pxe solution. In it core the pxe is much different from any other solution out there, what sets this a part this the way it done – with a little help from some custom scripts.

Because the organization mainly work in a Windows environment, the file that is used to kick of the process of updating the pxe with the latest info is a bat file – again this is only used to simplify the execution of the scripts need. So first execute the bat file which will execute a powershell script. This powershell script does two important things, first it get read two spread sheets, with network information about host management network and blade enclosure and second it logs into every blade enclosure get a complete inventory after which it creates a csv file with host information including Name,IP,UUID,Bay position and enclosure. This file gets uploaded to the pxe server and a custom bash script get executes locally on the pxe server. The bash script removes old boot menu files and kickstart files, after which it creates new once based on the csv file that got uploaded. All custom tailored to the blade it need to be used for. In order for this to work the blades UUID is used as the means for the pxe boot script to show the custom boot menu that will use the specific blades kickstart file, which as I mentioned before was tailor to the exact blade. The make the installation 100% seamless and automated.

The bat script only need to be executed where there are changes made to the over all setup, such as blade being physically move between enclosure, new enclosure be deployed etc.

The scripts

The bat file, this just ties it all to together, not much to say other then you need a way to do secure username/password, and it used plink.exe to communicate with the pxe server.

PXE_OA.ps1

This script does two important things, first it reads spread sheets for data and second is locks into all the OAs to get an inventory. The last part is some way overkill today, as there is a web server running on all OAs from where you can get all the same info without needing to login to every OA. But as I haven´t got around to correct it, this is still how its done in this version.

The script contains two functions which I didn ‘t make, one for importing excel spread sheet and one for doing tcp connections. Both of them deserve credit for there work, but It seems that I’ve shamelessly have copied just the part I needed and not the scripts in there full. For that I apologize, to whomever you are.

From line 160 and onwards there are things that need to be change to fit your given environment, please do so accordingly.

 

Run.sh

This is the last script which does the formatting and creation of custom files for the pxe solution to work as designed. Again there will be something that need to be change to fit your solution. This is some what easy to read so I wouldn’t go into to much detail – Line four is all about housekeeping, line 6-8 is about restructuring the UUID to fit with the format of the way the pxe uses UUID for boot menu and line sixteen does the customization of the kickstart config file in order to make every install unique.

 

The files

If you look at the run.sh script there is two files been copied repeatedly pxelinux.cfg and kickstart.cfg. pxelinux.cfg is the boot menu you see once the host pxe boots. The reason I am copying a file instead of just creating one from scratch is that it I can have a default static menu which is then customized with the dynamic entries needed. An example of a static boot menu option could be stress testing, but also firmware patching, ILO setup, bios settings etc. The last part is something I have done, using HPs Scripting ToolKit, but thats something for at different blog post.

The kickstart file, is the automation part of the ESXi install, it make sure that I don’t have to lift a finger during the installation and it guaranties the correct install and settings every time. The last part is what important, to have a simple, yet efficient way of guaranteeing a perfect install every single time! In my view that’s what pxe and kickstart is all about.

 

Pxelinux.cfg

Below is an example of how a boot menu is structured, the first five lines is global but then comes LABEL, which is what a selectable boot option is to be called, then comes how/what to boot and last some text which can give the menu option a meaning description. All this is static menu options, the run.sh script will then add the custom options for the ESXi install as that part isn’t static.

Kickstart.cfg

This is an early example, I’m not going to explain it, as it should be straightforward

Operational procedures

I been through a lot of configs and setup, but last I will show you how to create a custom build of ESXi to fit the need of your exact install and how to update the pxe server with a new ESXi install.

 

Image builder

The “script” below is for ESXi 5.1 and HP blades, and I have created it as generic as possible for ease of management. The first two lines add VMware and HPs esx depot to be used.

$ESXi variable has the lastes 5.1 version image in it

$Profile makes a writable copy of $ESXi, which we will use to customize the image

The next lines first installs all HP packages for 5.0, then remove all 5.0 package which are in 5.1 and lastly installs the remaining 5.1 packages. For some reason HP haven’t felt like updating the versioning numbers to fit with the different builds

The last to lines exports the custom image, first to ISO and then to zip (Bundle). The bundle can be load if changes needs to be made, just like with the online depots.

 

Update pxe image

The next part is the small changes there need to be done in order for a new image to work with the pxe, as there are comments below I’m not going to elaborate anymore.

 

Wrap up

This post became some what longer than expected and also took some what longer to finish then I planned. Just looked at the revisions of this post, I first started writing this post on the 10th of December 2013, hope it doesn’t show. But more so time has changed a lot since this solution was created. I still find it a good option, but I also see other good options which can make for a more complete solution. Which brings me to the things that a pxe solution can’t do… Which shortly put is everything which has to do with vCenter “services”, such as vDS, AD intergration etc. So these things need to be handle otherwise, which begs for at better way, that way could be vRO. But if vCenter “services” is not needed things like vSwitch can easily be part of the kickstart which would make installation complete.

 

Don’t think I have more to say at this point, but if you got this far, thank you for reading.

One thought on “Yet another ESXi pxe boot solution

Leave a Reply

Your email address will not be published. Required fields are marked *