Friday, December 03, 2010

Using Auto Installer on Solaris 11 Express

If you've ever had a conversation with me about Jumpstart, you know that I'm a big believer in the Jumpstart Enterprise Toolkit (JET). I've used it for over 7 years now and it's usually one of the first things I've setup for shops when I walk in the door to find that they don't have a good Solaris provisioning mechanism.

With OpenSolaris and thusly Solaris 11 Express, many changes have taken place around packaging, patching, and operating system installation. The one thing that has definitely changed things is Auto Installer, or AI as it's known. AI works with many of the same assumptions as IPS and SMF. No longer do you have pre or post installation scripts. You don't have all the options you've been use to using with things like JET or N1 SPS. However, this has been improving incrementally over the past year, but still has some way to go. I wanted to post what I've found to work.

The first step on this journey is to download the Solaris 11 Express AI images from:

http://www.oracle.com/technetwork/server-storage/solaris11/downloads/index.html

Once you've done that, you need to install the AI tools:

$ pfexec pkg install installadm
...

As you can see this service is dependent upon DNS, TFTP, and DHCP. If you don't have DHCP setup, it'll configure it for you. Or you can set it up with the following:

$ pfexec dhcpconfig -D -r SUNWfiles -p /var/tmp
Created DHCP configuration file.
Created dhcptab.
Added "Locale" macro to dhcptab.
Added server macro to dhcptab - katana.
DHCP server started.

$ pfexec dhcpconfig -N -m -t
$ pfexec dhtadm -M -m -e DNSserv=


If you have TCP Wrappers enabled, you'll have to open it up in your /etc/hosts.allow file with a "in.tftpd:ALL" line added to it.

Once that has been done, you can create what is known as an "install service" in AI:

$ pfexec installadm create-service -n s11-151a-sparc -s /export/install/isos/sol-11-exp-201011-ai-sparc.iso /export/install/aiserver/s11-151a-sparc
Setting up the target image at /export/install/aiserver/s11-151a-sparc ...
Registering the service s11-151a-sparc._OSInstall._tcp.local
Service discovery fallback mechanism set up
Creating SPARC configuration file

$ pfexec installadm create-service -n s11-151a-x86 -s /export/install/isos/sol-11-exp-201011-ai-x86.iso /export/install/aiserver/s11-151a-x86
Setting up the target image at /export/install/aiserver/s11-151a-x86 ...
Registering the service s11-151a-x86._OSInstall._tcp.local
copying boot file to /tftpboot/pxegrub.I86PC.Solaris-1
Service discovery fallback mechanism set up

What the first command will do is setup an installation service called "s11-151a-sparc" using the ISO we've downloaded and installing it into "/export/install/aiserver/s11-151a-sparc". The second command creates a service for x86 servers using similar syntax and of course the x86 ISO. This will enable SPARC clients to boot with WANBOOT and for x86 clients to boot with PXE. This will register the image with SMF install service, which you can check on below:

$ svcs -l install/server
fmri svc:/system/install/server:default
name Installadm Utility
enabled true
state online
next_state none
state_time November 27, 2010 02:03:25 AM CST
logfile /var/svc/log/system-install-server:default.log
restarter svc:/system/svc/restarter:default
contract_id 134
dependency optional_all/restart svc:/network/dns/multicast:default (online)
dependency optional_all/none svc:/network/tftp/udp6:default (online)
dependency optional_all/none svc:/network/dhcp-server:default (online)

This service will start-up Apache instances for each install service you create:

$ svcs -p install/server
STATE STIME FMRI
online Nov_27 svc:/system/install/server:default
Nov_27 1732 webserver
Nov_27 2164 webserver
Nov_27 2500 httpd
Nov_27 2571 httpd
Nov_27 2572 httpd
Nov_27 2573 httpd
Nov_27 2574 httpd
Nov_27 2575 httpd


You can see your AI service details with the installadm command:

$ installadm list
Service Name Status Arch Port Image Path
------------ ------ ---- ---- ----------
osol-b134-sparc off Sparc 46501 /export/install/aiserver/osol-b134-sparc
s11-151a-sparc on Sparc 46502 /export/install/aiserver/s11-151a-sparc
s11-151a-x86 on x86 46503 /export/install/aiserver/s11-151a-x86

At this point, your AI setup is very generic. So how do you customize it? Well you have to modify the default manifest for each service and create further manifests for any client specific customizations you want to do. Now, you can definitely read the documentation on doc.sun.com, which I would highly recommend considering this will continue to change.

However, to help you get started I've provided a sample default manifest you can use. The default manifest for each service is called default.xml, which is an XML file. The important thing about this file is that it'll be used as the base manifest and is critical to be careful with editing it, I would recreate a backup copy just in case before doing anything. So here it is:

<?xml version="1.0" encoding="UTF-8"?>
<!--
CDDL HEADER START

The contents of this file are subject to the terms of the
Common Development and Distribution License (the "License").
You may not use this file except in compliance with the License.

You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
or http://www.opensolaris.org/os/licensing.
See the License for the specific language governing permissions
and limitations under the License.

When distributing Covered Code, include this CDDL HEADER in each
file and include the License file at usr/src/OPENSOLARIS.LICENSE.
If applicable, add the following below this CDDL HEADER, with the
fields enclosed by brackets "[]" replaced with your own identifying
information: Portions Copyright [yyyy] [name of copyright owner]

CDDL HEADER END

Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.

-->
<!DOCTYPE auto_install SYSTEM "file:///usr/share/auto_install/ai.dtd">
<auto_install>
<ai_instance name="default" auto_reboot="true">
<target>
<target_device>
<swap>
<zvol action="create" name="swap">
<size val="4gb"/>
</zvol>
</swap>
<dump>
<zvol action="create" name="dump">
<size val="4gb"/>
</zvol>
</dump>
</target_device>
</target>
<software>
<source>
<publisher name="solaris">
<origin name="http://pkg.oracle.com/solaris/release"/>
</publisher>
</source>
<!--
By default the latest build available, in the specified IPS
repository, is installed. If another build is required, the
build number has to be appended to the 'entire' package in following
form:

<name>pkg:/entire@0.5.11-0.build#</name>
-->
<software_data action="install" type="IPS">
<name>pkg:/entire</name>
<name>pkg:/babel_install</name>
<!--
The following packages are required by iSCSI and included
by default to make it easier for users to enable iSCSI if
desired. They can be deleted from this list if iSCSI isn't
used. See iscsiadm(1m) man page for more information.
support for iSCSI.
-->
<name>pkg:/network/iscsi/initiator</name>
<name>pkg:/network/iscsi/iser</name>
</software_data>
<!--
babel_install and slim_install are group packages used to
define the default installation. They are removed here so
that they do not inhibit removal of other packages on the
installed system.
-->
<software_data action="uninstall" type="IPS">
<name>pkg:/babel_install</name>
<name>pkg:/slim_install</name>
</software_data>
</software>
<!--
Add missing driver packages to a booted install image so an
installation can complete. Add packages to target as well.
<search_all> searches and installs from configured repo.
-->
<add_drivers>
<search_all/>
</add_drivers>
<sc_embedded_manifest name="AI">
<!-- <?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type="profile" name="system configuration">
<service name="system/install/config" version="1" type="service">
<instance name="default" enabled="true">
<property_group name="user_account" type="application">
<propval name="login" type="astring" value="operator"/>
<propval name="password" type="astring" value="boajrOmU7GFmY"/>
<propval name="description" type="astring" value="default_user"/>
<propval name="shell" type="astring" value="/usr/bin/bash"/>
<propval name="uid" type='count' value='101'/>
<propval name="gid" type='count' value='10'/>
<propval name="type" type="astring" value="normal"/>
<propval name="roles" type="astring" value="root"/>
</property_group>

<property_group name="root_account" type="application">
<propval name="password" type="astring" value="boajrOmU7GFmY"/>
<propval name="type" type="astring" value="role"/>
</property_group>

<property_group name="other_sc_params" type="application">
<propval name="timezone" type="astring" value="US/Central"/>
<propval name="hostname" type="astring" value="solaris"/>
</property_group>
</instance>
</service>
<service name="system/console-login" version="1" type="service">
<property_group name="ttymon" type="application">
<propval name="terminal_type" type="astring" value="vt100"/>
</property_group>
</service>

<service name='system/keymap' version='1' type='service'>
<instance name='default' enabled='true'>
<property_group name='keymap' type='system'>
<propval name='layout' type='astring' value='US-English'/>
</property_group>
</instance>
</service>

<service name="network/physical" version="1" type="service">
<instance name="nwam" enabled="true"/>
<instance name="default" enabled="false"/>
</service>
</service_bundle>
-->
</sc_embedded_manifest>
</ai_instance>
</auto_install>


This will do the following:

  • Make clients reboot after the installation
  • Install the base Solaris 11 Express image
  • Install any drivers your platform requires that are not in the boot image
  • Create the "operator" account for doing administrative tasks using RBAC with the password of "newroot"
  • Set the root password to "newroot"
  • Set the timezone
  • Set the host name to "solaris"
  • Set the console type to "vt100"
  • Set the keyboard layout to "US-English"
  • Enable the NWAM service to auto configure the networking
Now this will enable your clients to come up with some account settings to get you logged in and set the language and console type. The client will grab the IP information from DHCP and call itself "solaris". But this hardly matches up with what Jumpstart could do for us. So now lets create a manifest for a client called the "test":

/export/install/aiserver/s11-151a-sparc/auto_install$ pfexec mkdir clients
/export/install/aiserver/s11-151a-sparc/auto_install/clients$ pfexec vi test.xml

<?xml version="1.0" encoding="UTF-8"?>
<!--
CDDL HEADER START

The contents of this file are subject to the terms of the
Common Development and Distribution License (the "License").
You may not use this file except in compliance with the License.

You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
or http://www.opensolaris.org/os/licensing.
See the License for the specific language governing permissions
and limitations under the License.

When distributing Covered Code, include this CDDL HEADER in each
file and include the License file at usr/src/OPENSOLARIS.LICENSE.
If applicable, add the following below this CDDL HEADER, with the
fields enclosed by brackets "[]" replaced with your own identifying
information: Portions Copyright [yyyy] [name of copyright owner]

CDDL HEADER END

Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.

-->
<!DOCTYPE auto_install SYSTEM "file:///usr/share/auto_install/ai.dtd">
<auto_install>
<ai_instance name="test" auto_reboot="true">
<target>
<target_device>
<disk>
<disk_keyword key="boot_disk"/>
</disk>
</target_device>
<target_device>
<swap>
<zvol action="create" name="swap">
<size val="4gb"/>
</zvol>
</swap>
</target_device>
<target_device>
<dump>
<zvol action="create" name="dump">
<size val="4gb"/>
</zvol>
</dump>
</target_device>
</target>
<software>
<source>
<publisher name="solaris">
<origin name="http://pkg.oracle.com/solaris/release"/>
</publisher>
</source>
<!--
By default the latest build available, in the specified IPS
repository, is installed. If another build is required, the
build number has to be appended to the 'entire' package in following
form:

<name>pkg:/entire@0.5.11-0.build#</name>
-->
<software_data action="install" type="IPS">
<name>pkg:/entire</name>
<name>pkg:/babel_install</name>
<!--
The following packages are required by iSCSI and included
by default to make it easier for users to enable iSCSI if
desired. They can be deleted from this list if iSCSI isn't
used. See iscsiadm(1m) man page for more information.
support for iSCSI.
-->
<name>pkg:/network/iscsi/initiator</name>
<name>pkg:/network/iscsi/iser</name>
</software_data>
<!--
babel_install and slim_install are group packages used to
define the default installation. They are removed here so
that they do not inhibit removal of other packages on the
installed system.
-->
<software_data action="uninstall" type="IPS">
<name>pkg:/babel_install</name>
<name>pkg:/slim_install</name>
</software_data>
</software>
<!--
Add missing driver packages to a booted install image so an
installation can complete. Add packages to target as well.
<search_all> searches and installs from configured repo.
-->
<add_drivers>
<search_all/>
</add_drivers>
<sc_embedded_manifest name="AI">
<!-- <?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type="profile" name="system configuration">
<service name="system/install/config" version="1" type="service">
<instance name="default" enabled="true">
<property_group name="user_account" type="application">
<propval name="login" type="astring" value="operator"/>
<propval name="password" type="astring" value="boajrOmU7GFmY"/>
<propval name="description" type="astring" value="default_user"/>
<propval name="shell" type="astring" value="/usr/bin/bash"/>
<propval name="uid" type='count' value='101'/>
<propval name="gid" type='count' value='10'/>
<propval name="type" type="astring" value="normal"/>
<propval name="roles" type="astring" value="root"/>
</property_group>

<property_group name="root_account" type="application">
<propval name="password" type="astring" value="boajrOmU7GFmY"/>
<propval name="type" type="astring" value="role"/>
</property_group>

<property_group name="other_sc_params" type="application">
<propval name="timezone" type="astring" value="US/Central"/>
<propval name="hostname" type="astring" value="test"/>
</property_group>
</instance>
</service>
<service name="system/console-login" version="1" type="service">
<property_group name="ttymon" type="application">
<propval name="terminal_type" type="astring" value="vt100"/>
</property_group>
</service>

<service name='system/keymap' version='1' type='service'>
<instance name='default' enabled='true'>
<property_group name='keymap' type='system'>
<propval name='layout' type='astring' value='US-English'/>
</property_group>
</instance>
</service>

<service name="network/physical" version="1" type="service">
<instance name="nwam" enabled="false"/>
<instance name="default" enabled="true"/>
</service>

<service name='network/install' version='1' type='service'>
<instance name='default' enabled='true'>
<property_group name='install_ipv4_interface' type='application'>
<propval name='name' type='astring' value='nxge0/v4'/>
<propval name='address_type' type='astring' value='static'/>
<propval name='static_address' type='net_address_v4' value='192.168.1.11'/>
<propval name='default_route' type='net_address_v4' value='192.168.1.1'/>
</property_group>
</instance>
</service>

<service name='network/dns/install' version='1' type='service'>
<instance name='default' enabled='true'>
<property_group name='install_props' type='application'>
<property name='nameserver' type='net_address'>
<net_address_list>
<value_node value='192.168.1.1'/>
</net_address_list>
</property>
</property_group>
</instance>
</service>

</service_bundle>
-->
</sc_embedded_manifest>
</ai_instance>
</auto_install>

This will set the host name and the static IP configuration, which is a bit more helpful than the default manifest. Now we must associate this manifest with the install service:

$ pfexec installadm add-manifest -m /export/install/aiserver/s11-151a-sparc/auto_install/clients/test.xml -n s11-151a-sparc -c MAC="00:14:4b:26:31:ab"
$ installadm list -m
Service Name Manifest
------------ --------
s11-151a-sparc test.xml


That will associate the manifest with the installation service and tie it to the MAC address of our client, which we haven't configured yet. You can use other ways of associating a manifest with an install service. But this is the easiest for associating it with a specific client.

Now lets create our client:

$ pfexec installadm create-client -e 0:14:4b:26:31:ab -t /export/install/aiserver/s11-151a-sparc -n s11-151a-sparc
Creating SPARC configuration file

$ installadm list
Service Name Status Arch Port Image Path
------------ ------ ---- ---- ----------
osol-b134-sparc off Sparc 46501 /export/install/aiserver/osol-b134-sparc
s11-151a-sparc on Sparc 46502 /export/install/aiserver/s11-151a-sparc
s11-151a-x86 on x86 46503 /export/install/aiserver/s11-151a-x86
$ installadm list -c
Service Name Client Address Arch Image Path
------------ -------------- ---- ----------
s11-151a-sparc 00:14:4b:26:31:AB Sparc /export/install/aiserver/s11-151a-sparc

At this point you can do an AI installation on the client:

{0} ok boot net:dhcp - install
Boot device: /pci@500/pci@0/pci@8/network@0:dhcp File and args: - install
/pci@500/pci@0/pci@8/network@0: 100 Mbps full duplex link up


The one thing that I've found that does not work in the manifest is the sizing of the swap and the dump datasets in the ZFS rpool. But this is easy to fix afterwards:

root@test:~# zfs set volsize=4g rpool/swap
root@test:~# zfs set volsize=4g rpool/dump


I'm sure I'll find more things to play with in AI. But I think you'll be able to learn from this and get started. Hopefully we'll see some plug-ins in JET and Ops Center next to support AI.

4 comments:

anthony11 said...

While reading the Solaris 11 Express document at http://www.oracle.com/technetwork/server-storage/solaris11/documentation/s11sysadminwp101109final2-186770.pdf I was alarmed by the statement " Note that the boot process for SPARC processor-based systems no longer uses Reverse ARP (RARP) protocol – instead, DHCP protocol is used regardless of the target machine architecture.", which led me here. How do SPARC systems without DHCP / wanboot support in OBP fit in?

Octave Orgeron said...

Basically, if you don't have wanboot support in your OBP, you're out of luck with AI and will have to do a DVD load instead. I think it's silly that AI has gone this route, but then again S11 EOLs the old XSun and the vast majority of the video cards out there. So even my SB2000 I can't use my Expert3D card on. At the same time, I can understand the need to retire support for systems over 7 years old.

ANKIT said...

Hi
I am trying to to pxe boot for sparc s11
here my questions and i need the help from experts.

I have 10GB adapter of XYZ company.
I create a local repo on my machine of solaris 11 b24.
This repo has a XYZ company 10Gb ethernet driver ,which is old driver and have some issues.
Now i want to replace this old driver with my new driver so that i can boot my client without any issue with this new driver.

Thanks in advance

ANKIT said...

Hi
I am trying to to pxe boot for sparc s11
here my questions and i need the help from experts.

I have 10GB adapter of XYZ company.
I create a local repo on my machine of solaris 11 b24.
This repo has a XYZ company 10Gb ethernet driver ,which is old driver and have some issues.
Now i want to replace this old driver with my new driver so that i can boot my client without any issue with this new driver.

Thanks in advance