RobC's Ruminations

SharePoint Site Collection Recovery

2010-05-28T10:41:00.000-07:00

This blog is a rework of a 'grok talk' presentation I gave in early may at the Cincinnati SharePoint Users group. I have used this same methodology to move or copy my SharePoint development site to new machines including from SQL Server 9.0 0 32bit to SQL Server 10.5 64 bit (although it wanted an upgrade to the database files before it would add them to the farm). It was suggested that all this could be set up with PowerShell. I would agree that this is possible. However, the first step would be setting up a PowerShell script to periodically capture the necessary data to drive the creation script. If I do this I will add a notice to this inro section.

SharePoint Site Collection Recovery

SCOPE

• DOES NOT

- Recover from a failed database server

- Farm scoped features that are not part of the Microsoft farm provisioning.

• DOES

- Recover/Move Existing SharePoint site collections. This includes user defines site collection columns and user defined site collection content types.

FAILURE SCENARIOS

• You hate ‘Guids’ in database names (and app pools in 2010) that the wizard creates!

• You need to bring up or move to a new box for some reason .

• SharePoint Server Failure.

Primarily with small farms without server redundancy.
Unintended disconnection from Farm and subsequent deletion of All Web apps by the ‘SharePoint Products and Technology’ wizard.
When uninstalling SQL Search Express 2005!
Brain was a weapon… AKA one click too many!

WHY DOES IT WORK

• Because the content of the site collections are stored in separate database(s) from the Farm database.

• Deletion of the web apps does not delete the web folders created under ‘C:\inetpub\wwwroot\wss\VirtualDirectories’. Specifically, this means the latest incarnation of each Webs ‘web-config’ still exists.

• Hence, to recover the site collection, one only needs to recreate the web app with properties (bindings & content database, etc) exactly like the original one(s) to re-attach them. This is possible because the ‘Create web app’ option allows naming an existing database.

• However, custom and third party ‘Farm Scoped Features’ are likely not recoverable just as they are not likely upgradable!

• There are some tables in the old farm database (DB) that can be copied to the new farm DB (such as Users) if It Is deemed more expedient because of number of entries, etc.

• The name of the new farm configuration DB does not have to match the old DB name. The others do.

ANTICIPATION

The following screen shots represent the minimum data required to rebuild the farm topology.

Periodically Save by Cut & Paste (may be done optionally by PowerShell function in future).

Use this page from Site Settings at the home level, to see the Site Hierarchy page

A View of What Was in IIS

It helps if the names contain the site collection name.

Reviewing Your Existing DB Names

Hopefully you edited the database name in the create web application form and used meaningful choices

Alternate Access Mappings

The Proverbial Hive

PREPARATION

The following steps MUST be executed in the order presented in order to avoid having to redo them because of failures such as databases not being empty or the wrong collation.

============================================================================

• Collect all screen shots, reports, together & order them basically in the order presented above.

• Using the ISS 6/7 management tool review and note all/any host header bindings and associated ports for each SharePoint web app.

• Using whatever tool you have, review the actual existing SharePoint database names & make any corrections to your printed copy.

• If you do not have a working directory at the 12/14 hive level then create one the called [Anything You Like] (Good for PowerShell Scripts As Well).

• Next create a shortcut for It. Move the shortcut to the desktop for expediency.

• Click on the shortcut and add a new text file called ‘build_farm.bat’.

• Again using your favorite editor, cut & paste the following text into the above File.

echo off
echo "Before you continue, make sure you have recreated your farm And administration databases "
echo "with the names as entered below And with a collation of 'Latin1_General_CI_AS_KS_WS'.“
echo "NOTE: this collation is specific to SharePoint And does not equal the default collation."
pause
..\bin\psconfig -cmd configdb -create -server [your servername] -database [WSS_Farm_Config] -user [user who is to be farm admin] -password [farm administrator’s password] -admincontentdatabase [WSS_Admin_Content]
pause

• Note that when using this script for the 2010 version, an additional parameter called “Passphrase” must be added to the command, where Passphrase is your choice of a string of 8 chars minimum.

• Using your favorite editor, cut & paste the following command into a file called ’C:\12hive.bat’ for example ( 14hive for 2010).

set prompt=$G
cd /D "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\[Whatever you named the new directory]“
dir

EXECUTION (The brown the meat section)

1. Installation

• If Installation of WSS/MOSS Is required (as on a new box) then do this Installation first.

• BUT when the completion screen Is displayed, UNCHECK the checkbox in the lower left that would run the ‘SharePoint Technology and Configuration Wizard’. We definitely do NOT want this to run until After we have created the farm manually.

2. Check Services

• It Is unlikely that any SharePoint services are running, but to avoid database lockouts review the running services and stop any service related to WSS and/or Moss (the Service Names will contain ‘SharePoint’ , ‘Office Server’, or ‘OWSTIMER’ , etc).

3. New Farm Database

• If one wants to use the same name for the farm configuration DB, rename or delete the current farm configuration DB.

• Using whatever database management tool you have, delete the current ‘Admin’ content database (DB).

• Now create the new farm DB remembering to change the collation from ‘default’ To ‘Latin1_General_CI_AS_KS_WS’.

4. Building the Farm Manually

• Using the ‘12hive/14hive’ shortcut locate the farm build ‘.bat’ script you created earlier in the new scripts directory.

• Open the script in your editor of choice and verify the arguments supplied for ‘server name’, ‘farm admin user name’, ‘admin password’, the farm DB name (just created), and your choice for the Admin content DB Name. Save the changes, then double click the .bat file.

• The execution of the ‘build.bat’ will execute steps 1-3 of 9 by PSConfig.exe. Thus provisioning the Farm and Admin DBs, resulting in a clean new farm configuration.

5. Provisioning the Farm

• The next step Is to locate ‘PSConfigUI.exe’ (aka: SharePoint Product And Technologies Wizard) in the ‘12hive\bin’ Directory With Windows explorer and double click on It. This will cause all 9 steps to be completed, bringing the Farm up in a state ready for the Admin to create web Apps and connect site collections to databases.

6. First Admin Steps

• This is where having the ‘What Was’ documentation from ANTICIPATION pays off.

• Open a console window. Enter ‘iisreset /noforce’. Press return and wait for IIS to reset.

• Now launch the ‘Central Administration’ web app from the ‘Administrative Tools’ menu.

• Ignore the whole Issue of completing Email, Servers, etc until later.

• Instead, select the ‘Application Management’ option.

7. Default Web App Definition

• From the ‘Site Hierarchy’ and/or ‘Alternate Access Mappings’, and ‘IIS Webs’ screen shots Identify the ‘Default’ zone web apps including their IP Ports, Host Headers and ‘managed paths’ if any.

• If the ‘default’ zone site collection requires a managed path, create it now.

• To save time in the next step determine the associated site collection name used previously from the page title of the ‘Site Hierarchy’ screen shot.

8. Create Default Zone Web Application(s)

• Then, for each ‘Default’ zone web use the ‘Create or extend Web application’ option to re-create the old Web App making sure to use the correct IP port, host header, content DB name and ‘App Pool’ name as before. (None of these parameters except the IP port number and content DB name is critical but is saves many correction steps later).

• NOTE: if you forget to change the content DB name then do only the first item of step 9 and then see ‘Appendix 2’ before doing the rest of step 9.

• If the web app create operation is fails and says to add the DB via ‘stsadm.exe’ see ’Appendix 1’ before continuing with step 9.

9. For Each Web Application

• If all goes well, after each web app is completed, in the console window enter ‘Up Arrow’ to get the ‘IISReset ’ command back and then press ‘Enter’ to execute the command.

• Next using the documentation and the ‘Central Administration->Application Management -> Create site collection’ option, create the site collection associated with this web app. At this point, using IIS, one should be to browse the whole site collection.

• Repeat Items 7 thru 9 for each Web App of the current zone.

10. For Each Zone Other Than ‘default’

• Next, tackle any ‘Intranet’ zone Web Apps by using the ‘Create or extend web Application‘ option in ‘Extend’ mode with the appropriate Port and Host Header from the documentation by repeating steps 7 thru 9

• At this point applying the previously defined ‘Alternate Access Mappings’ for the zone in question will enable testing browser access via the local Lan.

• Finally, extend the ‘default’ zone Web Apps to all previously defined ‘Extranet’, ‘Custom’, and ‘Internet’ zone web apps by repeating step 10 (which implies 7 thru 9) for each of the required zones in this step. If all the Web Apps for each zone were recreated to the correct port, the respective ‘web-config’s will be valid.

• Any third party or internal custom ‘Farm’ scoped features will have to be reapplied now.

SIDE NOTE:

• The main purpose of the ‘add content database’ page is to do literally as it says, add a another content data base to the site collection specified on order to allow SharePoint to spread the content load across databases based on the type of content (IE. PDF, office documents , pictures, lists, email, etc ) while the metadata may be intermixed

APPENDIX 1

• Create a new text file in the same scripts directory where you put the other ’ .bat’ files.

• Cut and paste the following command a new .bat file called ‘add-DB’ for example.

• Edit the file replacing the place holders in square braces with the IP port number of your web app , and content DB name.

• When correct, save the file and double click it.

• When successfully added proceed to Appendix 2.

Rem
..\bin\sysadm –o addcontentdb -url http://localhost:[wep app port #]
-databasename [the existing content db name]
pause

APPENDIX 2

• Using the SharePoint Administration Web App select the ‘Application Management>Content Databases’ option.

• Using the ‘Web application’ selection box at the top right of the inner panel select the web application you have just added the database to, either by the procedure in ‘Appendix 1’ or in step 8 above. If there is more than 1 database listed only the data base you want to keep show be showing a site count > 0.

• If there is more than 1 data base shown, select the one with the zero site count and it has a name you did not choose by clicking on the name. A new page will be displayed. First you must change the status to ’Offline’ and then select ‘OK’ so that the SQL Server does indeed take the database offline for safety.

• Now re-click on the same database name and this time when the same page is shown click the ‘Remove content database’ toggle button near the bottom the page. Then click ‘OK’. This remove the data base only as far as SharePoint is concerned. You will still need to delete the database in SQL server instance.

• When there is only 1 database listed and it has the name you expect then you are done here and can return to step 9.

2010-03-28T10:28:00.000-07:00

This is a copy of a short paper I wrote a couple of years ago. With the pending release of SharePoint 2010 the database sizes and other limits may get larger, however the underlying times and space consumption of 'restores' remains. To date there are no TechNet published timings for SharePoint 2010, as in the third reference for office 2007. If and when they show up, I will add the link to this post.

I hope this helps someones. RobC

SharePoint 'Job/Project' Topoloy Considerations

Foreward

Design of SharePoint topologies (hierarchy of site types) for any business with a ‘job’ or ‘project’ basis is a complicated process, as it is for any business. However, with some knowledge of the pertinent facts, the path to major decisions becomes a little more obvious. This document will try to present a concise collection of the facts and concepts that drive topologies in an AEC related world.

Many books on managing Moss/SharePoint installations mention choosing a ‘Publication Template’ for the initial site in the farm.

On the upside, a major benefit of having a publishing site collection at the root is, that it and all site collections of any template (except Blank templates) created below it will inherit the ‘advanced search’ capability,

On the downside, the ‘collaboration’ aspects of a ‘publishing’ site are non-existent and this is not good in a ‘job/project oriented scenario.

One important generality:

1. If one is just starting in SharePoint then start with ‘Foundation Server’ not WSS2.0 or WSS3.0. Updates ‘in place’ are usually ugly if you have modified something.

2. If you already have WSS3 convert NOW before you add complexity and content that possibly might make the conversion more troublesome.

So what to do?

Please read on and then evaluate.
Note: These conjectures are mine alone.

Decision 1 – Farm Implementation Type

• The ‘stand alone’ option should only be used in a SMB environment at best. There is No expansion possibility.

• A small farm can be implemented on one server like the standalone version or two or more.

• The single server small farm CAN be expanded to other servers by moving the web front and/or SQL server pieces.

• If the implementation is to be configured with separate SQL and WFE (Web Front End) server or the WFE servers are to be distributed on the WAN then the medium or large farm installation is a mandatory choice.

• One does not experience the management options that the ‘farm’ options provide in the ‘stand alone’ implementation.

• Summary: Always choose at least the ‘small farm option’.

Decision 2 – Primary Site Collection Template

• Publishing sites are normally used for content that is read only for the masses, especially for public facing sites, IE a portal with anonymous access and or an external site used across the WAN for your whole company.

• These publishing sites are commonly moderated, meaning: things must be approved via workflows or manually, by approvers before going public. This is not normally efficient or expedient on a job/project operational site.

• ‘Publishing’ Sites are typically used internally for division and/or departments to present company data to employees and management to support operations and ISO900?.

• Normally department/division management control the content of their respective sites via standard SharePoint built in tools via the browser.

• ‘Publishing’ site collections can only create child (sub) sites with a ‘publishing’ template, UNLESS one is creating a new site collection (as is done for the ‘MySites’ feature in MOSS implementations).

• ‘Team’ site collections and ‘team’ or ‘custom’ sub sites are the basis for real collaboration needed in a job/project operation.

• Site Collections allow putting all their shared content and that of their sub sites into a new database. This is important for back up & moving the site within the farm when scaling up or when servers fail.

• Site collections also provide inheritable attributes such as user permissions, presentation style and navigation to their sub sites.

• It is recommend by others in the field (Ref #2) that the size of any single content database in Not exceed 15-20 GB (SQL Server 2k5) and (from MS) no single SQL server instance exceed 32k databases and no single SQL server exceed 50 instances(ref #1). So per SQL Server databases is not a problem. Therefore a database per job is also not an issue (except for the naming ). Recovery May Be.

• With current (2007) disk drive technology a 15GB database takes about an hour to restore! And a little longer to save!!.

• However, if your database is living on a large SAN in a RAID 5/10 then recovery is not much of an issue. However, restoring archives may still be an issue of time.

Summary: Create a Publishing site collection at the root, especially when using MOSS.

Then use ‘Managed Path(s)’ to access team site collections where the managed path represents just plain ‘projects’ or some other logical hierarchy like state, Department, continent, managing office, etc .

Decision 3 - Projects per site Collection

Next question is: ‘How many projects per site collection? One, many, or a few.

Well, this question has no simple answer either. A review of the current document storage arrangement (like a disk based file or shared file system) will provide some insight into some practical constraints.

However, even these figures have to be viewed in the light of just what document types are currently being stored versus what additional document types are planned to be stored in SharePoint.

• An existing project might be considered small if the total storage required now is less than 2 Gigabytes (GB). Up to 8 of these projects could be under a site collection. But there should be some common ground like Division/Department, physical, or administration office. These factors would come into play under a possible site move from one server to another and apply to all site collections. Splitting the content data base on a site basis can be done via STP’s but this may not provide the exact division desired.

• Up to 10 GB could be considered a medium size project. This suggests about 2 sites of this size per Site Collection.

• Any single project with over 15 GB of current storage has to be considered a large project. Not from a SharePoint storage point of view, but from recovery one. Only one of these should be in a site collection as its size will lead to other complexities in sub topologies. To put that in some perspective, if an average CAD drawing size was 5 MB(Mega Bytes) then 15 GB could hold 3 Million of them (15* 10^12/5*10^6 = 3*10^6).

• Whether to use Folders or not in a given Library is a great question. In SharePoint 2003 they were not suggested as they were not handled well. In SharePoint 2007 it is a much different story. Especially where the Library content is 2000+ documents. See reference #3 and review the charts.

Decision 4 - Meta Data

One of the most important decisions pertains to just what is the minimum meta-data one needs to have per library to be efficient in searching and sorting the library content. There will be no shortage of content.

• If mixed document types (from the DOC, PDF, XLS, MSG etc. point of view) are to be contained in the same library then some careful thought about the meta-data is must.

• Are mixed content libraries a good thing. Even if isolated on a folder basis will this cause mass confusion.

• Would ‘naming conventions’ ease confusion and frustrations levels. -- Almost always, I think.

• The good news here is that usage of the ‘content types’ found in the 2007 version of SharePoint can help group or differentiate mixed content. For example all MSG and DOC type content could be combined under a defined content type of ‘communications’.

• A review of the present system (file share for example) as to how it’s set up, assisted (or did not assist) searches, and sizes, can lead to conclusions about what is required in the SharePoint environment.

The Case of the Missing 'Window1.xaml' file

2010-03-03T10:41:00.000-08:00

I had an issue when trying to debug an application (WPF) that complained about not finding the 'Window1.xaml' file. After Googling this, I saew lots of remedies some of which I tried to no avail. Most of these items referred to VS2k5 or VS2k8 betaX. I was using VS2k8 SP1 and had no problem with it Untill... after I had loaded VS2k10 beta2.
However the immediate solution was found after poking around a bit.

The simple solution is to examine the 'obj/app.g.cs' file, specifically the 'InitializeComponent()' routine.
The line staring with 'this.StarupUri = new System.Uri (' will likely only contain the name of your initial window 'eg '("Window1.xaml, System.Uri.Kind.Relative"). There is piece missing that should preceed the "Window1" word.
You can find/copy the exact values by editing your main window 'obj/Window1.g.cs' file and find the 'InitiailzeCompnent()' routine and you will see the required quoted value as the prameters to the 'System.Uri resourcelocator = new Syste.Uri(....)' call.

Once this value is pasted in to the approriate spot in the app.g.cs file you will be good to go.
One caveat, this fix will be undone by a 'clean, build/rebuld' operation beacuse the real root problem is not corrected (in my case something in the .csproj file).

Hope this helps someone recover quickly.

Sharepoint Site backup/archiving/cloning/moving

2009-08-14T10:45:00.000-07:00

While there are many ways in SharePoint to save/archive/backup/ an existing site and its subsites such as save as a template, part of a backup, or the new move/merge site option on Sharepoint Administration, one of the most flexable and sure method for the save/archive/backup is the 'export' option to 'stsadm.exe' found in the 12 hive's bin directory. Its couterpart the 'import' option fullfills the clone/move function.

The export option can export whole site collections. When doing sitecollections or sites with large files then increase the compressed file size with the'-cabsize NNNN' option.
On export operations set the '-Versions N' where 4 is most useful value for full backup and archive.
For clone operations '1' or '2' is probably the most efficient value depending on your library version settings. See the Reference section below.

The real beauty of this export is that not only includes contents but also the SD (designer) workflows. The worklows may have broken links on an import to a new site name but are easly fixed. At least they are there, unlike 'site templates'.

Since an exported site's contents can be imported on top of an existing site's contents, one gets the 'merge' operation. For this it is important that the -UpdateVersions option be set to '1' which adds them ias a new version where a list item or document exists, otherwise it is an error.

There are some cleanup things that should
Before doing an export for clone/archiving/move purposes one should take the time to:

delete all unnecessary copies of forms, datasources, et al created with SP Designer
if exporting a site collection retract and/or delete any unneeded solutions
empty all the recycle bins for all sites involved, both levels on site collection if enabled
for a clone/merge operations all unnecessary versions of list and documents should be deleted

On the export command it would be prudent to always include security in case its is reloaded from backup/archive. One can still chose to incude or igore the security on the import.

If one is doing a clone operation, E.G. creating a new department or office site then the security should NOT be included on the import command unless all sites should have equal access.

This abitlity to add content to an emtpy but pre-existing site is important when it is desired that the visibitly of the 'Top Link Bar' tabs and 'Quick Launch content be controlled by the user's role permissions. Manually created links are visible to all. Therefore, to get the default response one should create a blank site with the desired name using the browser tools and then do an import to this same site. Dependimng on your circumstances site Titles may have to edited unless you updated a pre-existing site.

References:

>stsadm -help export
stsadm.exe -o export -url

-filename [-overwrite] [-nologfile] [-quiet]

[-includeusersecurity] [-haltonwarning] [-haltonfatalerror]

[-cabsize

[-versions <1-4>

1 - Last major version for files and list items (default)

2 - The current version, either the last major or the last minor

3 - Last major and last minor version for files and list items

4 - All versions for files and list items]

>stsadm -help import
stsadm.exe -o import -url

-filename [-nologfile] [-quiet] [-haltonwarning]

[-includeusersecurity] [-nofilecompression] [-haltonfatalerror]

[-updateversions <1-3>

1 - Add new versions to the current file (default)

2 - Overwrite the file and all its versions (delete then insert)

3 - Ignore the file if it exists on the destination]

stsadm -o export -url http://server/sites/abc -filename .\savefile -includeusersecurity

-nologfile -versions 2

stsadm -o import -url http://server/sites/ghi -filename .\savefile -nologfile -versions 1

Happy site management.

Missing Fields in SharePoint Event Property Lists

2009-08-04T21:36:00.000-07:00

Ever wonder why some fields that one would expect to be included, do not show up in before and/or after properties in the Sharepoint event properties. It should have been obvious, but it took the little brain within a night or two to sus out the idea that it was the ones with 'null' values that were not in the before list, and therefore definitely not in the 'after list.
So if you need the field to be included for an update during a synchronous ('ing') event make sure it has a 'default' value.
Note that if one one turns on 'Manage Content Types', the metadata fields that are in the event properties are limited to those defined in the content type (I am referring to the fields added by the project users not the fields defined by Microsoft such as created/modified by, file leaf ref, etc. ). When mutiple content types are defined for a library and one has logic in an event receiver that always expects to test the value of that field then it must be defined in all content types or a 'not in range' exception will occur (don't you just love these helpfull exception messages!).