Server 2008 System State Backups

In Server 2008 they removed ntbackup and replaced it with Windows Server Backup Utility. Unfortunately this doesn't backing up the systemstate to a network share anymore. In addition you can't backup to volumes listed as critical volumes (ie the OS volume).

First off, you can disable the inability to backup to critical volumes: http://support.microsoft.com/kb/944530/en

With this method you could backup to your C: drive and the use scheduled tasks to copy the backup to another location.

wbadmin start systemstatebackup -backupTarget:C: -quiet

OR

If you have access to an iSCSI SAN you can give the server a new drive attached to the SAN which it will detect as a valid local disk. At this point you can dump the backup to this volume (which isn't local so less risk of loss) and then backup using your remote backup solution.

Note that for Windows 2008 R2 the commands are different

Also plan ahead for the size of the backup. I found that my total System State backup on a 2003 server was only 700MB, but on my 2008 servers it increased to over 10GB!

Citrix Access Gateway Virtual Appliance on XenServer

Thanks to ervik for this! http://www.ervik.as/index.php/citrix-mainmenu/xenserver/1885-how-to-run-citrix-access-gateway-cag-on-citrix-xenserver

I'm placing my version of it here for my ease of access and also because I found loading his site very slow.

XenServer v5.5
CAG v4.6.1

1. Create new VM
2. Other Install Media
3. Use the CAG iso for install media
4. Add 512MB RAM
5. Add Virtual Disk 12GB (appears that you can get away with as little as 5GB)
6. Add 2 NIC's (must have 2)
7. It will do it's thing and then display to remove install media and reboot
8. Remove the iso image and reboot
9. When it displays "Adding IPv4 address 40.30.20.10 to the eth1 interface..." your done
10. Give your management workstation an additional ip of 10.20.30.x
11. Navigate to https://10.20.30.40:9001 and install the admin tools
12. Username: root Password: rootadmin

Server 2008 "Preparing your Desktop" stalls

Recently I built a new Server 2008 SP2 Domain Controller. When logging into it the "Preparing your desktop" would stall for a long period of time. The machine itself wouldn't freeze and functionality continued normally and you could even use the task bar, just not the desktop.

Fix:
Open cmd prompt and type: Net localgroup Users Interactive /add

Logoff and back on with no more stalling.

Manage Temporary Internet Files with Group Policy

Temporary Internet Files should be proactively managed to help reduce security risks. In addition it can help with other issues as well, such as the Outlook attachment opening issue: "Can't create file: filename. Right-click the folder you want to create the file in, and then click Properties on the shortcut menu to check your permissions for the folder." http://support.microsoft.com/kb/305982

• Get the Group Policy Client Side Extensions for all of your machines. http://support.microsoft.com/kb/943729
• Depending on your machine SP level you may also need to install XMLLite. Check out this site for a list of requirements depending on the SP level: http://blogs.technet.com/grouppolicy/archive/2009/03/27/group-policy-preferences-not-applying-on-some-clients-client-side-extension-xmllite.aspx
• Both can be pushed using your favorite method (third party, group policy, manually, etc)
• At this point you can begin to push the new Group Policy objects
• Open Group Policy Management (note that you cannot manage these new GPO's from Windows XP, they can only be managed from Vista, 7, or 2008)
• This can be done several ways depending on your preferences. I did it by computer role / operating system.
• In Active Directory I have all Terminal Servers in one OU, Workstations in another OU, Laptops in another, etc. As such it made sense to link the GPO by the computers role and operating system
• It is important (to some extent) that the operating system be specified with this. For instance XP and Vista do not have the same paths to the Temporary Internet Files
• Windows 2000, XP, 2003 = C:\Documents and Settings\%LogonUser%\Local Settings\Temporary Internet Files
• Windows Vista, 7, 2008 = C:\Users\%LogonUser%\AppData\Local\Microsoft\Windows\Temporary Internet Files
• User Configuration - Preferences - Windows Settings - Folders
• New Folder - Replace - proper pathing to TIF location (depending on which OS you are targeting)
• Check the following:
• "Recursively delete all subfolders"
• "Delete all files in the folder"
• "Allow deletion of read-only files/folders
• "Ignore errors for files/folders that cannot be deleted"
• Common Tab - Item-level targeting
• Targeting Button - New Item - Operating System
• Set the operating system (notice that you can add multiples and right click it change the AND to OR, for instance if you want it to read Windows Server 2003 OR Windows Server 2003 R2)

Ensure that you have the GPO linked to the proper Active Directory OU and that if you link it to an OU with computers in it rather than users that you enable loopback policy - merge.

Migrate Print Server with Server 2008

Below is a guide on how I moved my Windows Server 2003 print server to Windows Server 2008 SP2 with little to no end user interaction.

With Windows Server 2008 there are a lot of new Group Policy options that are really cool! These help tremendously in helping manage an enterprise and this is what you'll use to change printers (and even set defaults).

• Build your new Windows Print Server (in my case Server 2008)
• Extract printers from old print server (in my case Server 2003)
• From Print Management console
• Right click old print server and select Export Printers to a File. **In my case I had some corruption issues with 2 ports that caused this to fail. To get around this I used the printmig tool to export and import my printers initially. Upon import with printmig it told me which ports where at fault so I removed them from the new server and all associated drivers/printers and recreated them.
• Right click new print server and import printers from a file (file you just created)
• At this point you should have 2 print servers
• Get the Group Policy Client Side Extensions for all of your machines. http://support.microsoft.com/kb/943729
• Depending on your machine SP level you may also need to install XMLLite. Check out this site for a list of requirements depending on the SP level: http://blogs.technet.com/grouppolicy/archive/2009/03/27/group-policy-preferences-not-applying-on-some-clients-client-side-extension-xmllite.aspx
• Both can be pushed using your favorite method (third party, group policy, manually, etc)
• At this point you can begin to push the new Group Policy objects
• Open Group Policy Management (note that you cannot manage these new GPO's from Windows XP, they can only be managed from Vista, 7, or 2008)
• Come up with a map of which Active Directory OU's will need different printer configurations. There are a lot of cool ways to do this by specifying specific OS's, Users, Computers, etc
• Create a new (or edit an existing) GPO with a link to the desired OU
• Under preferences, Control Panel Settings you'll find Printers
• NOTE: notice that there is a column for Order.... you get the idea, they are applied in order
• Right click, new, Shared Printer (for Shared networked printers)
• At this point you'll have options for Create, Replace, Update, Delete
• Set the first one for action Delete, check box "Delete all shared printer connections"
• Common tab, check "apply once and do not reapply" and "Item-level targeting"
• Click Targeting
• Enter the criteria you want to target (ie Computer, User, Security Group, Operating System). I targeted Operating System with the overall GPO linked to an OU of the department that the users work in. This way it hits the user object for any computer of that operating system they log into. For instance All accounting users are in one OU so they all get this specific setting when they log into a Windows XP workstation (thus it doesn't affect their Terminal Server printers unless I set it to).
• Right click, new, shared printer again
• This time select create and path to the printer (\\servername\printersharename)
• Set as default if desired
• Common tab, "Apply once and do not reapply"
• Item-level targeting again as you desire
• Continue for each printer

NOTE: if you link the GPO to an OU that has computer accounts rather than user accounts then you will also have to enable loopback policy - merge (under computer config - Policies - Admin - system - Group Policy - User Group Policy loopback processing mode)

At this point as long as the computers have the Client Side Extension and the GPO's are linked properly they will get the old printers removed and new ones added / default set

Once all users have a policy setup for them and all CSE's are deployed to workstations you can zap the old print server (notice that you can Log Spooler information events to see if anyone is still printing to the old server)

Scenario:

• User group of 10 needs 2 printers Printer1 and Printer2
• 9 users need printer1 to be the default. 1 user needs Printer2 as default
• All 10 users are in the same active directory OU
• You don't want to split them into new OU's

1. Create Delete All printers policy (apply once only option)
2. Create Printer 2 policy (apply once only option)
3. Create Printer 1 policy and select the set as default (apply once only option)
4. Update Printer2 policy and select the set as default (apply once only option). Important: Item-level targeting - New Item - User - Enter the user. Should read "the user is domain\username (SID match)". This could also be done by computer or however you desire

Transfer DHCP from Server 2003 to Server 2008

Microsoft has really done a great job making this process easy and smooth.

http://support.microsoft.com/kb/962355

• On 2003 DC open cmd prompt
• Netsh
• dhcp
• server file://servername/
• export c:\w2k3DHCPdb all
• copy to new DC
• Log onto new DC and open command prompt
• net stop dhcpserver
• rename or delete DHCP.mdb from system32\dhcp folder
• net start dhcpserver
• netsh
• dhcp
• server file://servername/
• import c:\w2k3DHCPdb
• Restart DHCP and verify that all information was moved
• Change your scope options to the correct settings if needed (ie DNS and WINS)

Don't forget to Deactivate the old server scope(s) that where transfered.

Install Windows Server 2008 SP2 Domain Controller

Install a Windows 2008 SP2 Domain Controller in a Windows 2003 Forest / Domain

Prep Forest

• run netdom query fsmo from Domain Controller to ensure Schema Master. Note the holder of the following roles
• Schema owner
• Domain role owner
• PDC role
• RID pool manager
• Infrastructure owner
• On the Domain Controller running the Schema Owner do the below:
• Attach the W2K8 DVD to Schema Master
• Navigate to D:\sources\adprep
• Run adprep /forestprep
• Type C and Enter to continue
• Allow Replication to forest before continuing
• On each DC open adsiedit.msc (note that it will state which DC it's connected to in the OU)
• Navigate to Configuration - ForestUpdates - ActiveDirectoryUpdate
• Open properties
• Check Revision attribute and should be set to "2"
• http://technet.microsoft.com/en-us/library/cc771922(WS.10).aspx

Prep Domain

• Note from previous the Infrastructure Owner
• Run the following on the Infrastructure Owner
• Attach the W2K8 DVD
• Navigate to D:\sources\adprep
• Run adprep /domainprep /gpprep
• Allow replication through forest before installing Domain controller
• On each DC open adsiedit.msc (note that it will state which DC it's connected to in the OU)
• Navigate to Domain - Root - System - Domain Updates - ActiveDirectoryUpdate
• Open Properties
• Check Revision attributes and should be set to "3"
• http://technet.microsoft.com/en-us/library/cc770385(WS.10).aspx

Install 2008 Server Domain Controller

• Server Manager
• Roles - Add Role
• Check Active Directory Domain Services
• Install
• Close wizard and launch the active direcotry domain services installation wizard
• Existing Forest - Add a domain controller
• Current credentials
• Next
• Yes to the adprep /rodcprep (read only) warning
• Select site / next
• Leave DNS Server and Global Catalog checked - Next
• Default locations
• Enter password for Restore Mode
• Next
• Reboot on completion check
  
  Transfer FSMO Roles
• Note that MS recommends moving FSMO roles when rebooting Domain Controllers
• Log onto the DC that will be getting the FSMO roles assigned to it (ie the new one)
• Click Start - Run - type ntdsutil and press enter
• type the following commands
• roles
• connections
• connect to server *servername* (name of the server you want to roles transfered to)
• q
• transfer schema master
• transfer naming master (note that I found that this role has different tranfer names between Server 2003 SP2 R2 and Server 2008. In 2003 it was domain naming master, in 2008 it is naming master)
• transfer PDC
• transfer RID master
• transfer infrastructure master
• q
• q
• From command prompt type netdom query fsmo and check that they are on the correct domain controller

Install other management as needed

• WINS
• Note that WINS is a feature not a role in Server 2008
• Determine if you need WINS server installed
• Open Server Manager
• Click Features - Add Feature
• WINS Server select
• Install
• DHCP
• Ensure you have setup a Static IP (should have with the start of Domain controller installation)
• Server Manager
• Click Roles - Add Role
• DHCP Server
• Select NIC
• Check domain and dns server address
• Enter alternate address if desired (can do later as well)
• Enter WINS address if needed
• Enter Scopes or leave blank (I prefer blank and setting up later)
• Disable DHCPv6 unless you need it
• Enter credentials
• Install
• Don't forget to go back and configure your DHCP options later

SysPrep Windows Server 2008

Basic Server 2008 Sysprep

• Navigate to C:\Windows\System32\sysprep
• Run Sysprep.exe
• Check "Generalize" (this regenerates SID's)
• Change to Shutdown
• Bingo, done, easy.

Slipstream W2K3 SP2

Extract files needed

• Download the ISO's for W2K3
• Download the network installs for SP2
• Mount the ISO for W2K3 and copy the files to a new folder (ie C:\W2K3)
• Run from command prompt the SP2 file name ie SP2.exe /x
• Extract to an easy location ie (C:\SP2). Command for this would look like SP2.exe /x:C:\SP2

Slipstream SP2

• Navigate to C:\SP2\i386\Update and enter update.exe -s:C:\W2K3
• The slipstream will start
• Convert the W2K3 folder back into ISO form

XenServer iSCSI SR's not connecting

The other day we had an issue with one XenServer VM's hanging so I attempted to do a force reboot. Unfortunately I found that this failed (never timed out). After attempting to run xe task-cancel uuid=xxx on the task in question and that not working something that I had found said to run xe-toolstack-restart (DO NOT DO THIS). The toolstack restart failed misureably and led me to having to restart the host. Once it was back online the iSCSI fun began.

• The restarted host was the master
• It showed 'almost' all SR's as broken to include the local DVD drive
• SR's that didn't show broken still couldn't be booted from
• Broken SR's couldn't be repaired successfully

This led me to start looking at the iSCSI SAN which is an HP Lefthand Networks SAN/iQ v8.1. After opening the SAN/iQ management console I found that many of the Snapshot schedules I had setup where 'paused' due to backlog. In addition all the SS's that I had deleted where still listed, but already reported as deleted if I attempted again.

Things to note:

• The week prior one LH node had the RAID controller card fail and had to be replaced
• The failed card had been replaced and system powered back on so that it could restripe
• All VM's run off the LH Cluster that had the failed LH node in it
• Snapshots wouldn't delete from either of the 2 clusters in the LH setup (VM cluster or Storage cluster)
• Gateway connections to the XenServer host showed in a 'failed' status

The LH rep very quickly pointed out that the Local Bandwidth Priority was set to .25 MB/sec. Yikes! That's not right. Changing this setting back to the recommended 4 MB/sec helped a 'little', but not very much. We then changed it to 10 and the difference was not much better. Fortunately I did notice that within a few minutes the XenServer host has picked up it's SR's again. YEAH!

As I was waiting for things to replicate so that speeds would pick up again on the network (I had set it back to 4MB/sec by this point) it occured to me that the node with the failed RAID controller would be attempting to resync still along with all the Snapshot data.

BINGO! Shutdown the LH node that had failed and instantly everything picked up and ran at lightning speeds again. XenServer kicked in and all admin tasks worked great again. Once everything was connected and all Snapshots where taken care of I turned back on the failed LH node and let it resync which was fairly quick at this point and caused no more heart failures.

Leasons learned:

• .25 MB/sec is way to slow for admin tasks on LH nodes (I already knew this, but now I know to check it)
• Backlogged LH admin tasks can cause the iSCSI connection initiations to slow to a crawl (I was told this shouldn't affect it, but imo it clearly did).
• Don't run xe-toolstack-restart unless you absolutely have to. I could have easily fixed the root of the issue (LH replication) without the outage had I not run this command
• After major failures such as the RAID controller check up on it periodically to ensure that it's finished / processing in a timely manner. Had I done this I would have found the Snapshot issue and resync backlog days in advance.

NIC Intermittent Connectivity

We have an older desktop provisioned for use by a user with an application which is "less than friendly". So rather than mess with the TS environment we gave her the old desktop. Recently she started having issues with slowness which progressed into lots of messages about Outlook retrieving data, extreme slowdown, network drives dropping offline, and other clients losing connection.

I quickly found in the eventvwr that the tcp/ip connection was going up and down every few minutes. After changing the patch cable and testing the desktop on a new network drop I found the answer. Changing the speed of the NIC from "auto detect" to "100 Mbps Full Duplex" resolved the issue. Apparently in it's old age something started causing it to fail to negotiate the speed. As such it was constantly trying to re-negotiate the speed which caused the up / down connections.

Install SharePoint to share port 80

I decided that I wanted my Sharepoint site and my MediaWiki site on the same server. MediaWiki was already installed and using port 80, but I don't want my users having to type in an address with a port on it (they would simply just not use the site if they had to remember the port). Unfortunately websites can't share ports on the same ip address. Plus, I didn't want to move the wiki site off port 80 either.

I did the following on Windows 2008 IIS.

1. Install SharePoint to port 80 (in my case as the non-default website)
2. Open Manage network connections (ncpa.cpl from run)
3. For your network connection go into properties
4. Go into IPv4 properties
5. Assumtion is that you already have it set to Static IP address
6. Click Advanced
7. Click add and add in another unused IP address (ex: 192.168.1.100 for main and add in 192.168.1.101)
8. Ok out
9. In IIS click the default website
10. On the right side click Bindings
11. Edit and change the IP address from * to the main IP address (192.168.1.100 in our example)
12. Okay out
13. Click the new Sharepoint site (Default name is SharePoint - 80)
14. Click Bindings and change IP address to the secondary IP (ex 192.168.1.101)

From here you can access the original website normally and the Sharepoint site via the new IP address. This brings up some new issues though...

DNS Entry to make the site "friendly"

1. Open your domain DNS and add a new Host (A). Make the Name what you want your users to type in to reach the site, then enter the second ip address (SharePoint site address)
2. Try to ping the name you just entered. It should pingback as the ip address you just set.
3. Try to navigate to the site (ex: http://example/)

Next issue... Authentication loopback check doesn't like this setup much. At this point you'll find that credentials fail. After a lot of searching I found the solution here: http://blogs.bluethreadinc.com/thellebuyck/archive/2008/10/30/401.1-error-when-accessing-sharepoint-from-server.aspx

1. Click Start, click Run, type regedit, and then click OK
2. Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. Right-click Lsa, point to New, and then click DWORD Value
4. Type DisableLoopbackCheck, and then press ENTER
5. Right-click DisableLoopbackCheck, and then click Modify
6. In the Value data box, type 1, and then click OK
7. Exit Registry Editor
8. Restart the computer.

Note that the auther includes the following: "The security is reduced when you disable the authentication loopback check, and you open the Windows Server 2003 server for man-in-the-middle (MITM) attacks on NTLM."

Virtual Iron > XenServer Convert Data Disks

A majority of my Volumes on servers in Virtual Iron are setup using the MS iSCSI Initiator rather than having made them as Vitual Disks within VI. This made most of my conversions very easy. I had 1 case where a disk other than the system volume was a VI Disk. To move this disk to XenServer here's what I did.

1. Use XenConvert 2.0 on the machine with the disk attached (ensure that any services such as SQL etc have been disabled so that the files on the disk are not in use)
2. From: This Machine (machinename)
3. To: Xen Virtual Appliance
4. Choose disk (D: in this case)
5. Choose location
6. Once it's finished go to XenServer and select import VM
7. Setup the import like a normal VM, but at the end deselect the "Start automatically" option
8. Once finished importing you can delete the VM but choose to leave the SR intact
9. Go to the server that needs the disk attached (or import if you haven't already) and attach the disk that's needed

Restore XenServer with HP Lefthand Networks SAN

Today I had to restore a XenServer VM with the SR residing on an HP Lefthand Networks iSCSI SAN (SANiQ v8.1). It was smooth as butter and made me all happy inside due to the ease of the restore ;)

I had created a Wiki site for internal admin use on a W2K8 server on IIS. I decided that I didn't want to build another W2K8 server and use another license for the SharePoint site so instead I decided to have it run on the wiki site as well. During the install I made the mistake of creating the SharePoint site as the default site which was very effective for killing my Wiki site.

Luckily I had created a Snapshot on my LH Networks SAN prior to the SharePoint install. Here's the steps I used to restore.

1. First I gave my test XenServer pool access to the Snapshot in the Lefthand Console.
2. Turned off the Production server
3. Detached the SR for the Production Server in question
4. Created a new SR in the test pool
5. Target IQN of the Snapshot name
6. When you click finish it will see the disk and warns not to attach if other pools are using the SR. (thus the reason we turned of and detached the production server / SR). Click Yes
7. Create a new vm with correct properties.
8. Select any install media, it won't matter as you won't be installing
9. You won't be able to select the Virtual Disk you want since it won't have free space. So just select any disk and we'll fix later
10. Give it a nic
11. UNcheck the Start VM automatically
12. Go into the properties of your new VM
13. Change boot order so HD is first
14. Go to Storage and Attach the correct Virtual Disk
15. Delete the Virtual Disk from when you created the VM (if you selected one)
16. Ensure RAM / CPU are set correctly and boot.
17. Check over the server to ensure it's what you want to restore

Once I verified that this was the server snapshot that I wanted I went to pull this into production

1. Turn off the test server you just created
2. Forget the Virtual Disk (this doesn't destroy data)
3. Go into the HP Lefthand console and and right click the Snapshot you want and choose "Rollback".
4. All Snapshots and changes created after that snapshot will be lost! Make sure this is what you want first.
5. Go back to XenServer Console and click the Production SR and click Attach.
6. Fill in the IP info and Discover LUN etc. Click yes to the warning about other VM's on it again.
7. Start server
8. Glance around the office to see if anyone noticed that the wiki was down ;)

View Network Connections when Control Panel icons hidden

Occasionally I come across computers where the Control Panel has been "locked down" and the icons are hidden. Usually the hidding of these icons is done via a Group Policy.

I've found a couple of times now where the Network Connections icon is hidden, but this can be useful for finding out of a computer is getting an IP address correctly or not. Sure the command prompt ipconfig works too, but if the admin has the Control Panel locked down then surely they have the command prompt locked down ;)

Click Start
Click Run
Type ncpa.cpl
You'll see the friendly network connections your used to seeing (Windows XP).
Right Click the connection and select status, support tab, details.