A Technical View of Borland MIDAS Part II
(Multi-tier Distributed Application Services)
(Version 1.3)
Copyright © 1997 by Charlie Calvert
Building a Remote Data Client Application
Now that you have the server set up, the next thing to do is set up the client. In this example, I will assume you are running both the server and the client on the same machine. Once the design phase is over, you can move the server to a remote machine, as covered later in the paper.
Here is quick overview of the steps involved in creating the client:
- Drop down a TRemoteServer and use its ServerName property to connect it to a server. These servers are listed in a drop down combo box. Set Connected to True.
- Drop down a TClientDataSet control and set its RemoteServer property to the server you just wired. Set the ProviderName property to the name of a dataset provider from the server. These providers are listed in a drop down combo.
- Hook up database controls to the TClientDataSet just as you would were it a TTable or TQuery.
- Now run your application.
I will now go over these steps again in more detail. To get started, create a new application, and drop down a TRemoteServer object on it, and fill out its properties as shown in Figure 4.
Figure 4
: Filling out the properties for the TRemoteServer component.
When hooking up the TRemoteServer component, the key property is called ServerName. You can use the ServerName drop down combo box property editor to find the items you need. If these fields are blank, or do not contain the name you need, then the server is not registered on your current system. To remedy the situation, run the server once on your client machine.
The act of running the server places an entry in the Windows Registry. To see the entry, run RegEdit.exe from the Run menu item on the Start menu. Open up HKEY_CLASSES_ROOT and search for the name of the executable that contains the server. For instance, the key in this case is called CustOrdersServer.CustOrdersRemoteData. The first half of this key is from the executable name of the server, and the second half is from the name you gave to the remote data module, which is also, by default, the name of the exported COM interface.
If you want to access a server on a remote machine from a Delphi TRemoteServer, you can use the ComputerName property to locate the machine. Once again, there is a special dialog that pops up when you select ComputerName, so that you can select the computer by name.
* * * Begin Note * * *
Note: In this example, you might want to simply connect to a remote server located on the current machine. To do this, simply run the server once, and it will be registered and listed in the drop down list for the ServerName property. To access the server on a remote machine, run it once on the local machine, then once on the remote machine, and then access that machine through the ComputerName property. If you have DCOM services set up properly on both machines, then everything should work automatically. If you want, you can use DComCfg.exe to point DCOM at the remote machine when running your server, or you can use OLEnterprise to do the same thing. You can, if you want, use some other means than running the server to make entries in your registry. There will be more information on connectivity, DCOM and OLEEnterprise later in this paper.
* * * End Note * * *
Now drop down two TClientDatasets on a form. Use the drop down list on the RemoteServer property to hook them both up to RemoteServer1. Use the drop down list on one TClientDataSet to hook it up to the CustomerDataSet on the server via the ProviderName property. Use the ProviderName property on the second data set to hook it up to the OrdersDataSet on the server. When you use the ProviderName property the Connected property on the TRemoteServer object will be set to true and the remote server will be loaded into memory.
Set the client Dataset's Active property to true, and hook up some data sources and grids to them. You can then use the MasterSource and MasterFields properties to establish a one to many relationship between the two tables. You have now completed the client application. You can run it, and see your tool in action.
* * * Begin Note * * *
Note: Particularly when working across machines, I think it is often a good idea to launch the client first, and then connect to the server. Here is code that would connect to the server at run time:
CustomerClientDataSet.Active := True;
OrdersClientDataSet.Active := True;
Or, if you prefer:
CustomerClientDataSet.Open;
OrdersClientDataSet.Open;
The act of setting the Active property of a dataset to True will automatically set the Connected property of the TRemoteServer to True.
I recommend connecting at run time for two reasons. One, the launch of your application will be relatively quick, and will be sure to succeed. Two, there is no need to connect to the server every time you run the application. As I said earlier, the client can save the data set to disk, and then allow the user to edit without connecting to the server. Obviously, you can't use this feature properly if you don't give the user a choice when it comes time to connect to the server.
* * * End Note * * *
The Briefcase Model
The briefcase model depends on two methods of TClientDataSet called LoadFromFile and SaveToFile:
CustomerClientDataSet.SaveToFile('Customer.dta');
CustomerClientDataSet.LoadFromFile('Customer.dta');
If you have two tables you want to save, then you should save and read them both to separate files:
procedure TForm1.BriefcaseSave1Click(Sender: TObject);
begin
CustomerClientDataSet.SaveToFile(CustomerFile);
OrdersClientDataSet.SaveToFile(OrdersFile);
end;
Clearly the act of saving and reading files from disk is trivial in the extreme. However, there are some additional points that you need to take into consideration to use the briefcase model properly. Most of these points will be covered in the next section on PacketRecords.
PacketRecords
PacketRecords is a very important property of TClientDataSet which you need to spend some time contemplating. In the next few paragraphs I will discuss it from several angles. In particular, I will make several references to its importance when using the briefcase model.
To make the briefcase model work correctly it is probably best to make sure the files on the server are not arranged in a one to many relationship and that you have set the PacketRecords property on both the ClientDataSets to -1. Setting PacketRecords to 0 brings down the metadata, setting it to -1 brings down all the data, and setting it to some positive number n brings down n records per request.
If you have already gotten the metadata for a dataset, then setting PacketRecords to -1, or to some positive number other than zero, will only retrieve data. However, if you have not gotten the metadata, then setting PacketRecords to -1 or to some positive number other than zero, will retrieve both the metadata and the records.
If you want to use the briefcase model, then you usually want to bring down the whole dataset, which means you want to set PacketRecords to -1. (Of course, a query on a server could already filter a large portion of a table for you, but you would still want to set PacketRecords to -1 to retrieve all of the records from the query. )
If you just want to establish a one to many relationship, and don't care about the briefcase model, then you will probably want to set PacketRecords to zero on the detail table and to -1 on the master table. These settings retrieve all the records from the master table but only the metadata for the detail table. Then internally, Delphi will call TClientDataSet.AppendData to bring down just those detail records that are needed when viewing one particular master record. This is great for many situations, but it is probably not what you want if you are using the briefcase model. Instead, when using the briefcase model, you will usually set PacketRecords to -1 and the master detail will still be done, but the whole detail dataset will be available on the server at all times. This is obviously impractical when working with very large datasets.
Because this is such an important issue, I am going to show you how to write some code that allows you to fine tune this process. The following code represents a nonsensical case where you first retrieve the metadata and then retrieve the records. I say this is nonsensical because the metadata will be retrieved automatically the first time you access the data. However, assuming you had some reason to get the metadata first and then get all the records, you could write code that looks like this:
with ClientDataSet1 do begin
Close;
PacketRecords := 0;
Open;
PacketRecords := -1;
GetNextPacket;
end;
Or, if you wanted to be very fancy, you can study this code from Delphi guru Josh Dahlby:
var
RecsOut: Integer;
V: OleVariant;
begin
CustomerClientDataSet.Close;
V := CustomerClientDataSet.Provider.GetMetaData;
CustomerClientDataSet.AppendData(V, False)
V := CustomerClientDataSet.Provider.GetRecords(-1,RecsOut);
CustomerClientDataSet.AppendData(V, True);
end;
In this case you first close the client dataset, then use the GetMetaData function to retrieve the metadata inside an OleVariant. At this time, there is not much you can do with the result of this function other than pass it directly to AppendData, or else start hacking away at it with a series of low level functions. However, in the future, it is reasonable to suppose that Borland may provide functions for easily accessing the information from the data packet that is stored in the OleVariant.
AppendData takes two parameters. The first is the data retrieved from the server, the second is whether or not you hit EOF when retrieving the data. Remember that you don't have to use either GetRecords or AppendData, and that you should normally use GetNextPacket. Furthermore, the simplest way to perform this operation is simply to call Open or to set Active to True. I've shown you GetNextPacket and AppendData simply so you can have more control over the process if you happen to need it.
Updating and Refreshing Data
If you have edited one or more records and want to make the changes permanent. Then you should call ApplyUpdates.
CustomerClientDataSet.ApplyUpdates(-1);
Passing -1 to this method means that you want the update process to stop when an error occurs. ApplyUpdates returns the number of errors it encountered.
To understand what is happening here you have to understand that Delphi caches all the changes you make to a dataset. In other words, it keeps both the original record, and the updated record. When you call ApplyUpdates, errors can be reported, and you will have the chance to revert to the original record, or to attempt to push your changes through. Error handling is explained in the next section.
If you have successfully updated the server, then you will probably also want to refresh your data set with any changes made by other users. To do this, call Refresh.
Here is a typical example of how the whole process might look in action:
if CustomerClientDataSet.ApplyUpdates(-1) = 0 then
CustomerClientDataSet.Refresh;
if OrdersClientDataSet.ApplyUpdates(-1) = 0 then
OrdersClientDataSet.Refresh;
Error Handling
When working with remote data sets, there are going to be occasions when errors occur. For instance, if two users are accessing a table at the same time, then it is possible that they will both want to change the same record. In this case, the person who first performed the update would succeed in changing the record, while the second person would get an error.
Errors are passed back to a TClientDataSet and can be handled by responding to the OnReconcileErrorEvent. A detailed explanation of responding to errors would take up almost as many pages as I've written so far, so I will consider that topic beyond the scope of this paper. However, there is a simple solution to this whole problem that can be covered in just a few paragraphs.
The trick to handling errors returned from an Application Server is to use a form stored in the Delphi Object Repository. To find the form in question, select File | New from the menu, turn to the Dialogs page, and opt to Copy the Reconcile Error Dialog. Save the dialog in the same directory as your current project, and remove it from the files that are automatically created at start up. To do this, choose Project | Options | Forms from the Delphi menu.
Add the Reconcile Error Dialog to the uses list of the appropriate form in your project. In many cases, this will be the main form for your project. Now add the following in response to an OnReconcileError event:
procedure TForm1.CustomerClientDataSetReconcileError(
DataSet: TClientDataSet; E: EReconcileError; UpdateKind: TUpdateKind;
var Action: TReconcileAction);
begin
Action:=HandleReconcileError(Dataset, UpdateKind, E);
end;
This one line function will launch the dialog you found in the Object Repository and give the user the ability to handle any errors, as shown in Figure 5.
The grid in the center of the form tells the name of the field on which the error occured. The Modified Value is the value the client application wanted to insert into the record. The Conflicting value is the value the "other guy" who beat you to the update inserted into the record. The Original Value is the value the record had before either update was made. As you can see, the user has the option to Skip, Cancel, Correct, Refresh or Merge the data.
Needless to say, you can make all these changes, and access all these options by writing your own code. However, it probably makes more sense just to use this dialog, or to use the dialog as the basis for your own code.
That is all I want to say about the process of creating a client or server application. The rest of this paper will deal with connectivity issues. Remember that a pair of programs called CustOrdersClient and CustOrdersServer accompany this paper. If you are having trouble finding these examples, you can look for them on my web site: users.aol.com/charliecal.
Connecting Remotely
What you can expect in the next few pages is a brief overview of DCOM and of OLEnterprise. In particular, I will concentrate on trouble areas, and on giving you the key information you need. Additional details are available on my web site (users.aol.com/charliecal) in the paper on DCOM.
DLLs Needed on Each Side
When you are installing on servers and clients, there are several files that are needed on both sides.
Server Side files that are needed:
- All the DLLs from BDE4 have to be on the server.
- DBCLIENT.DLL
- IDPROV32.DLL
- STDVCL32.DLL
DBCLIENT and IDPROV32 are only included in the client/server version of Delphi, and to get them on your system you need to answer yes to the license agreement screen in the install.
DBCLIENT and STDVCL32 need to be registered, that is entered in the registry. You can register them with REGSVR32.EXE from Microsoft or with the TRegServ Delphi demo.
DBCLIENT.DLL is the only DLL needed on the client. STDVCL32.DLL is not required but it is likely to be needed. It is the type library for IProvider and IDataBroker. DBCLIENT needs to be registered on the client. If it is on the path, then it will be registered automatically when your application loads.
Using DCOM
If you are running a default installation of Windows NT, then you don't have to do anything to install DCOM. I assume that the next version of Windows 95, currently called Memphis, will provide the same service for you.
If you are running Windows 95, then you need to download DCOM for Windows 95 from Microsoft's web site. At the time of this writing, you can do this by signing on to Microsoft's web site at www.microsoft.com, and then heading for the "Free Downloads" section. While you are there, be sure to get DCOMCfg for Windows 95.
Setting up DCOM for Windows 95 is trivial, in that all you have to do is run the install programs you downloaded from the website. However, getting DCOM to work correctly on either Windows 95 or Windows NT is more complex. In particular, most users will need to set up a Domain before this process will work correctly.
If you do not care about security, then you can connect without setting up a domain. In particular, if you sign on to a Windows 95 machine and to a Windows NT machine with the same name and password, then you can make a connection without setting up a domain.
Using Windows 95 as a DCOM Server
If you want to use Windows 95 as a server, you will loose some security and several additional ease of use features. This article will be updated to reflect information on using Window 95 as a DCOM server, but for now you can rad the overview of this subject in this section and check out the following article:
http://www.microsoft.com/kb/articles/q165/1/01.htm
(Article ID: Q165101)
The first two paragraphs of the article read as follows:
"To use a Windows 95 computer as a Distributed Component Object Model (DCOM) server, you need to configure the computer to run DCOM applications. Using a Windows 95 computer as a DCOM server will not provide the same features as a Windows NT 4.0 computer. Performance and security will be more limited. Additionally, you will have to perform some steps manually that are provided automatically by Windows NT. For instance, you will probably want to add shortcuts to the server components in your startup group and you will need to run RPCSS.EXE before you will be able to access the server component.
Because you need to have the server component running before trying to use the client, you need some mechanism to keep the server running. The easiest way to do this is to have a blank form in the server. You may set the visible property of the form to 'False.'"
Another key section of the article suggest you take the following steps:
"Change the key below in the Windows 95 registry:
HKEY_LOCAL_MACHINE\Software\Microsoft\OLE\EnableRemoteConnect
to the following:
Y
Failure to set this key results in the following error on the client:
"Run-time error '429': ActiveX component can't create object"
Run DCOMCNFG on the Windows 95 Server. Select the server component and click the "Properties" button. Select the "Location" tab and ensure that the "Run application on this computer" check box is selected.
Select the "Security" tab. Select either "Use default access Permissions" or "Use custom access permissions." If you opt to use custom permissions, click the Edit button.
Use the "Add Access Permissions" dialog box to grant or deny access to users of the server component. Click OK to close the "Add Access Permissions" dialog box."
Almost all of this section is simply a quote from the MS article, and I will add a more detailed description at a later date. I will assume, however, that most users will want to set up an NT domain. This is a complicated subject, so I will discuss it in the next section of this paper.
Here is a little additional information that I have not followed up on, but which may be of use to some developers: If you want to use peer-to-peer, or a Novell network, and don't have access to an NT server, you can use share-level access and change the registry setting:
LegacyAuthenticationLevel=1
For more details, see:
http://www.microsoft.com/kb/articles/q158/5/08.htm
The details on this article are:
INFO: COM Security Frequently Asked Questions
Last reviewed: July 3, 1997
Article ID: Q158508
An excellent source of information, current at the time this article was written, is Dan Miser's website:
http://www.iinet.com/users/dmiser/dcom95.htm
Setting up a Domain
DCOM is relatively useless unless you have a domain server. Domain servers are regular install of Windows NT that are set up to work with domains. You cannot change a standard install of Windows NT Server into a domain server. You must reinstall the server, being careful this time to configure the server as a domain server, not as a stand alone server.
All the Windows NT work stations and Windows 95 machines that you want to use your distributed application should now become members of your domain. To make that work properly on a Windows 95 machine you should open the Network applet from the Contol Panel and turn to the Access Control page. Make sure you have User Level Access Control and not Share Level Access Control. This means security will be handled by downloading a list of users and their rights from the domain server. In short, you have to have a domain server in the network or this process will not work correctly.
While still in the Network applet, flip to the Configuration page, double click on Client for Microsoft Networks, and state the name of your domain server. (In this discussion I am omitting correct configuration of network services such as TCP/IP.)
If you are new to domain servers, you will probably find them a bit confusing at first. The best way to get up to speed is to buy a book on the Windows NT Server. There are a number of these books available on the market, and some of them do a good job of explaining the intricacies of domains. My suggestion is to browse through a bookstore until you find one that contains a good explanation of domain servers. If you read their explanation in the book store and it seems incomplete or incoherent, then it probably won't get much better when you go back home. Browse around until you find at least a reasonably competent book, as this subject is just tricky enough to warrant a good treatment from a third party book.
Using DCOMCfg
When you are using DCOM, there are two different ways to make the connection. You do not always have to be conscious of the difference between the two techniques, or even that the two techniques exist, but it is best if you do have at least some knowledge of the technology involoved.
To make a connection to an object on a remote machine, the system needs to know both the name or IP address of the remote machine, and the GUID of the object that you want to connect to on that machine. (If you have the prog id of an object, you can get the GUID by calling the ProgIDtoClassID function from the ComObj unit. This process can be handled for you automatically by the IDE.)
It turns out that you can put entries in the registry that will allow windows to access an object on a remote machine as if it were on the local machine. What happens is that you call the object the standard way for handling local objects, but when Windows looks up the object in the registry it sees that the object actually resides on a remote machine and takes the necessary steps to access it. As far as the user is concerned, the object appears to be local, but behind the scenes Windows is making the connection remotely.
The low level code called by the IDE is propably similar to the CreateOleObject call used by Delphi users to connect to local COM objects and the CreateRemoteComObject call made by Delphi users to connect to remote objects. Once again, you don't have to call these methods, since the IDE does it for you, but I mention them in case you want to dig into this technology further, as explained in the DCOM paper found on my web site.
You can, of course, edit the registry yourself in order to make the changes necessary so that Windows can treat a remote object as if it were a local object. However, the simplest way to make the changes is by using a program called DCOMCFG.EXE. This program ships with Windows NT, and is available for Windows 95 via Microsoft's web site.
DCOMCFG, can also be used for myriad other purposes, most of which involve security. You can use this utility to force a server on a Windows NT machine to appear on the screen, rather than run in the background.
Using OLEnterprise
OLEnterprise is an alternative to DCOM. If you use OLEnterprise, then you do not need to have DCOM on your system.
OLEnterprise has several advantages over DCOM:
- It will allow you to make connections between two Windows 95 machines, even if no NT server is available. Connections without a server is impossible under DCOM. Connections between two Windows 95 machines even when a server is present are either impossible or extremely problematic under DCOM. In particular, DCOM only seems to work reliably when you are placing servers on NT machines. You can connect from a Windows 95 machine to an NT machine with DCOM, but not vice versa, nor can you reliably connect between two Windows 95 machines. (As far as I can tell it is impossible to place a server on a Windows 95 machine for remote connections, but I am hedging a bit here since this area is so poorly documented that I may have missed a trick hidden in some obscure location. Also, I suppose that this state of affairs will change as DCOM matures. At any rate, I have never spoken to any one who has made the connection successfully, though many very talented programmers have tried.)
- OLEnterprise has an Object Broker that will allow you to distribute the load of connections over multiple machines. In particular, each time a new user signs on to a database, they can be randomly routed to an available server, thereby distributing the load over multiple servers. The act of replicating the data between servers is, quite naturally, not supported by OLEnterprise, but is instead the responsibility of the server itself.
- OLEnterprise has fail over capability available through writing a few lines of code.
Here is the fail over example, as provided by the good graces of the indefatigable Mike Destein:
procedure TForm1.Button2Click(Sender: TObject);
begin
try
ClientDataset1.ApplyUpdates(-1);
except
on eOle : EOleException do begin
Case eOle.Errorcode of
-2147023169 : // I dont know the const name
begin
// Handle RPC failure by resetting server
RemoteServer1.Connected := False;
RemoteServer1.connected := TRUE;
ClientDataset1.ApplyUpdates(-1);
end;
else showMessage(intToStr(eOle.errorCode));
end;
end;
end;
end;
Using OLEnterprise
When installing OLEnterprise on a Windows 95 machine, do not install into the default directory, but instead install off your root, and try to avoid long file names in the directory path. Any problems with OLEnterprise and long filenames on Windows 95 machines will be corrected in future releases of the product. To work around any potential problems, simply install into a directory structure that does not use long filenames.
There are four key pieces in the OLEnterprise toolset:
- The Object Broker, called Broker.exe, and it should be run on the server machine before loading the Object Factory. If you start it with a -D option it will spew out debug information that can help you understand that process of connections, and help you confirm that connections are occurring in the correct order and at the correct time.
- The OLEnterprise Configuration utility, called OLECFG.exe. This utility is used to specify whether or not you are using an Object Broker, and what machine the Object Broker is running on. Remember that the broker's job is to distribute connections randomly, as such it is not needed in order to make the connection. It is a helpful utility, but not a necessity. Therefore the configuration gives you the option of turning it off. You should run the configuration utility on both the client and the server.
- The Object Factory, called ObjFact.exe. This utility is the core of the system, and you must run it on the server or OLEnterprise will not work. However, you do not have to have this utility running on the client to make a connection.
- The Object Explorer, called OLEntExp.exe. This program plays a similar role to DCOMCFG.exe. It allows you to browse the objects available on your system, to export objects from a server, and to import objects from a remote system. You should remember that the explorer uses the registry as the main repository of information about the objects on a system. If you have run a server once on a machine, the explorer will find the object in the registry, and you can then use the menu to export or import the object. If you are on a client machine, and are using the explorer to search for programs on remote machines, then you must be sure that you have used the OLEnterprise Configuration utility to specify where the object broker you are currently using is running. If you launch the explorer, and it takes a very long time to come up, or it appears to hang at times, then it is likely that the machine is searching for the broker but cannot find it. The program can take several minutes before it allows the search to time out and returns control to the user. The same thing will happen if you click on the Object Broker option in the program's left panel. One click, and a search for the broker will begin. If the broker is not found, use the configuration utility to specify its location, or else go to the remote machine and make sure the broker is running. If you start the broker with the -D option, then you should see output from it when you start an Object Factory on the same machine, or when you try to connect to it from a remote machine.
When using the Explorer, you need to learn how to import and export objects. This is an intuitive process accomplished by manipulating the menus or by right click on items in the main panels of the application. You should probably spend some time using the default sample programs shipping with the product to make sure you understand importing and exporting objects. In particular, go to etc\samples\auto\memoedit to find a good pre-built sample program.
Summary
In this paper you have had a look at Borland's multi-tier technology. In particular, you have seen how to create servers and clients, and how to use DCOM and OLEnterprise to connect to a remote server.
This technology is important for several reasons:
- It provides a means of creating thin clients that make few demands on the client system.
- It simplifies, in fact nearly eliminates, the need to configure the client machine.
- It allows you to partition applications in logical compartments. If you wish, each of these compartments can be run on a separate machine, thereby distributing the load of the application.
- It provides a means for distributing a load over several server machines, or for routing the load to a specific machine with the power to handle heavy demands. The previous bullet point covered breaking the app up into the server, application server and client, while this bullet points covers the advantages of using multiple servers in conjunction with the object broker.
- It provides a robust architecture for handling and reporting errors, particularly in a multi-user environment.
- It allows you to use a briefcase technology that stores files locally, then allows you to reload them when it is time to update the server. This is ideal for laptop users who spend a lot of time on the road.
For many users, this technology is so compelling that it entirely replaces the standard database architectures used in Delphi 2. These users are attracted to the ability to partition the applications into logical pieces, even if the entire application is being run on a single machine. However, the biggest benefits achieved by this architecture become apparent when you bring multiple machines, and even multiple servers, into play.
MIDAS
