A few weeks ago we posted two tips on pulling HTML from a URL. (Here's part 1 and part 2). Someone asked us about web pages that are secured, requiring a user name and password. So we've updated our Java code to allow for an optional user name and password.

The user name and password are optional, so updating the Java code won't require updating any LotusScript code using the Java code. The way we've implemented this is by setting up variables, initially empty strings, to hold the user name and password. If the variables are empty strings at the time the URL is read, then nothing extra is done. If the variables have been set through a new method added to the class (setUserNameAndPassword), then those will be used to access the web page. The updated Java here still goes into a Java library, just like the previous tips:

import java.io.*;
import java.net.*;
import java.util.*;

public class GetHTML {
   
   private Vector result = new Vector(); // An array of each line of HTML
   private String userName = "";
   private String password = "";
   
   public void readHTML(String urlToRead) {
      URL url; // The URL to read
      HttpURLConnection conn; // The actual connection to the web page
      BufferedReader rd; // Used to read results from the web page
      String line; // An individual line of the web page HTML
      try {
         url = new URL(urlToRead);
         conn = (HttpURLConnection) url.openConnection();
         conn.setRequestMethod("GET");
         if ( (userName.equals("") == false) && (password.equals("") == false) ) {
            String userPwd = userName + ":" + password;
            String encoding = new sun.misc.BASE64Encoder().encode(userPwd.getBytes());
            conn.setRequestProperty("Authorization", "Basic " + encoding);
         }
         rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
         while ((line = rd.readLine()) != null) {
            result.addElement(line);
         }
         rd.close();
      } catch (Exception e) {
         e.printStackTrace();
      }
   }
   
   public int numLines() {
      return result.size();
   }
   
   public String getHTML(int pos) {
      if (pos < result.size()) {
         return (String)result.elementAt(pos);
      } else {
         return "";
      }
   }
   
   public void setUserNameAndPassword(String u, String p) {
      userName = u;
      password = p;
   }
}

Refer to the earlier tips for descriptions of the methods in this class. The new method added is "setUserNameAndPassword". You pass in the user name and password as strings and they will be used when the page of HTML is read. Remember, calling this method is optional.

The LotusScript changes a little bit, but only if you want to use the user name and password. If you are reading a "open to the public" web page, you don't need to change your LotusScript code at all. Here's some updated LotusScript to access the Java class and read the secure web page. Refer to part 1 to get more details on calling Java within LotusScript.

Const myURL = "http://www.breakingpar.com"
Dim js As JAVASESSION
Dim getHTMLClass As JAVACLASS
Dim getHTMLObject As JavaObject
Dim size As Integer
Dim i As Integer
Dim html As String
Dim userName As String
Dim password As String
   
Set js = New JAVASESSION
Set getHTMLClass = js.GetClass("GetHTML")
Set getHTMLObject = getHTMLClass.CreateObject
userName = "user"
password = "pwd"
Call getHTMLObject.setUserNameAndPassword(userName, password)
Call getHTMLObject.readHTML(url)
size = getHTMLObject.numLines() ' Get the total vector size (elements 0 to size-1)
For i = 0 To size-1
   html = getHTMLObject.getHTML(i)
   Print html
Next