In a heavily-used application, with a lot of adding and deleting documents, and a lot of client replication, we found that some of our LotusScript routines were returning invalid documents (deleted documents, for example) from our NotesView.GetAllDocumentsByKey and NotesDatabase.Search methods. Obviously, this could lead to errors (the "Document Has Been Deleted" error was our favorite). The database simply couldn't keep up with all the view index changes, but our LotusScript routines needed to ignore the deleted documents (and conflict documents).

We ended up writing a Filter function that we would use all the time. So instead of:
Set coll = view.GetAllDocumentsByKey(key, True)
our code became:
Set coll = Filter(view.GetAllDocumentsByKey(key, True), "Subject")

It was similar for the NotesDatabase.Search calls that we were using in the application.

This Filter function takes two parameters. The first parameter is the collection you want to filter. The second is either a string or an array of strings. This is the name of the field (or names of all the fields if an array is passed) that must exist on EVERY document in the collection for it to be a valid document. In the example above, we were passing in the single string "Subject", so every document in the returned collection will have a field called "Subject".

One thing nice about the function is that you can be assured that the return value will be a valid collection. It might be empty, but it will be valid. To have bullet-proof code, after getting a collection from a search or through a view, you should really check to see if the collection object is Nothing or not. If the collection is Nothing, then it won't have a Count property and you won't be able to get the first document, so your code will error out with an Object Variable Not Set error. This function always returns a valid collection object.

The function filters out documents that are deleted, documents that are conflicts, and documents that are missing the Form field. In preparation for using this function, you'll want to create a new view called Empty in your database. The view should have a selection formula of SELECT @False. This will assure that there's nothing in the view. If you don't want to add that empty view to your database, you can still use this function but it will be a little slower. We'll talk about that after the code:

Function Filter(coll As NotesDocumentCollection, mustHaveFields As Variant) As NotesDocumentCollection
   Dim view As NotesView
   Dim retColl As NotesDocumentCollection
   Dim doc As NotesDocument
   Dim nextDoc As NotesDocument
   
Notice that the 2nd parameter is defined as a Variant. This is needed so either a string or an array of strings can be passed into the function.

   ' Initialize the return collection to an empty collection
   If coll Is Nothing Then
      Dim session As New NotesSession
      Dim db As NotesDatabase
      Set db = session.CurrentDatabase
      Set view = db.GetView("Empty")
   Else
      Set view = coll.Parent.GetView("Empty")
   End If
   Set retColl = Nothing
   ' Make sure we get a valid object to return
   While retColl Is Nothing
      Set retColl = view.GetAllDocumentsByKey("A key that will not exist", True)
      If retColl Is Nothing Then Call view.Refresh
   Wend

The first bit of code establishes a valid object that will be returned. This means that any calling code does not have to make sure the object is valid before proceeding. It can be assured that it is dealing with a valid object. If you don't want to use the view, then you can perform a NotesDatabase.Search here. Your search string should be SELECT @False (or something where you can guarantee that no documents will be returned). So the retColl statement would be changed to:
      Set retColl = coll.Parent.Search("SELECT @False", Nothing, 0)

Note that doing a search all the time is going to be slower because Notes is going to have to build an index each time instead of using an index that is already built in the view.

   ' Make sure we start out with no documents in the return collection
   If retColl.Count <> 0 Then
      Set doc = retColl.GetFirstDocument
      While Not doc Is Nothing
         Set nextDoc = retColl.GetNextDocument(doc)
         Call retColl.DeleteDocument(doc)
         Set doc = nextDoc
      Wend
   End If

The view should be empty, so the collection should have nothing in it at first. But here we make sure. If there happens to be something in the collection initially, that would throw off our results. So this block removes everything from the collection.

   ' If an invalid object was passed in, then return the empty collection
   If coll Is Nothing Then
      Set Filter = retColl
      Exit Function
   End If

Just in case the first parameter is an invalid object, return a valid object with no documents.

   ' Go through the documents in the original collection. For those that are valid,
   ' put them into the collection to be returned
   Set doc = coll.GetFirstDocument
   While Not doc Is Nothing
      If IsValidDoc(doc, mustHaveFields) Then Call retColl.AddDocument(doc)
      Set doc = coll.GetNextDocument(doc)
   Wend
   ' Return the updated collection
   Set Filter = retColl
End Function

Finally, go through all the documents and find out which ones belong in the collection. This involves calling the "IsValidDoc" function, which is shown next:

Function IsValidDoc(doc As NotesDocument, mustHaveFields As Variant) As Integer
   Dim i As Integer
   
   If doc Is Nothing Then
      IsValidDoc = False
   Elseif doc.IsDeleted Then
      IsValidDoc = False
   Elseif doc.UniversalID = "" Then
      IsValidDoc = False
   Elseif Len(doc.UniversalID) <> 32 Then
      IsValidDoc = False
   Elseif doc.UniversalID = String$(32, "0") Then
      IsValidDoc = False
   Elseif doc.HasItem("$Conflict") Then
      IsValidDoc = False
   Elseif Not doc.HasItem("Form") Then
      IsValidDoc = False
   Else
      IsValidDoc = True
      If Isarray(mustHaveFields) Then
         For i = Lbound(mustHaveFields) To Ubound(mustHaveFields)
            If Not doc.HasItem(Cstr(mustHaveFields(i))) Then IsValidDoc = False
         Next
      Elseif Cstr(mustHaveFields) <> "" Then
         If Not doc.HasItem(Cstr(mustHaveFields)) Then IsValidDoc = False
      End If
   End If
End Function

This function takes a document object and the same 2nd parameter as the Filter function. The function checks all the possibilities that we've found in our years of programming in Notes. So the document must meet all these criteria before being called valid:

• It must be a valid object (must not be Nothing)
• It must not be a deletion stub
• It must have a UNID
• The UNID must be 32 characters in length
• The UNID must not be all zeros
• The document must not have a $Conflict field
• The document must have a Form field
• If the 2nd parameter is an array, then all the field names listed in the array must be present in the document
• If the 2nd parameter is a single string, then that field name must be present

If all the above conditions are met, then the document is valid. Note that if you don't care about any additional fields (more than the Form field being present) you can pass in an empty string as the second parameter.

If you want to reuse this code, you'll have to decide if you want to eliminate conflict documents (like our code does) or not. We found that processing the conflict documents ended up causing more problems. For example, if a document should have 3 responses then the document having a coflict (which is a response) ends up throwing the count off. We use other processing to find conflicts and then either report them to our administrators or eliminate the conflict if it doesn't need to be handled manually.