A Technical View of Borland MIDAS
(Multi-tier Distributed Application Services)
(Version 1.3)
Copyright © 1997 by Charlie Calvert
Overview
In this paper you will learn about multi-tier computing as implemented in a Delphi-hosted technology called Borland MIDAS. Multi-tier computing allows you to partition applications so you can access data on a second machine without having a full set of database tools on your local machine. It also allows you to centralize business rules and processes, and distribute the processing load throughout the network.
This paper is divided into four parts:
- An introduction to multi-tier computing
- An Overview of Borland’s multi-tier Computing Technology
- A detailed look at creating servers and clients
- A detailed look at connecting remotely
Throughout this paper I will use the terms multi-tier computing, distributed datasets, remote datasets, and Midas as approximate synonyms. Later in the paper I have a section on terminology, which explains these words and their significance. Check for updates to this paper on both www.borland.com and on my Web site: users.aol.com/charliecal. Note that throughout this paper, as in all my writings, I use repetition to emphasize and clarify key points.
An Overview of Borland’s Multi-tier Computing Technology
In this section I make a few broad statements about multi-tier computing. My goal is to give a general definition of the technology, particularly for the sake of users who are new to the subject. In later portions of the paper, the discussion will be increasingly technical and increasingly specific.
Though distributed datasets can be based on DCOM, there are parallels between multi-tier computing and the Web. In particular, a person can use a browser to view a dataset on a remote machine without having any database tools on the client. The same is true of Borland’s multi-tier implementation of distributed datasets.
Unlike Borland distributed datasets, browsers are severely limited in terms of functionality. For instance, without a powerful third-party tool such as IntraBuilder, it is difficult to enforce constraints, to program a browser to perform a join, or to set up tables in a one-to-many relationship. These chores are simple to execute inside a Delphi multi-tier application. Delphis high performance compiled applications are also much faster and much more responsive than HTML-based applications.
Distributed datasets allow you to use all the standard Delphi components, including database tools in your client side applications, but the client side does not have to include the Borland Database Engine, ODBC, or any client database libraries (e.g., Oracle SQL*NET, Sybase CT-Lib, etc.) Somewhere on the network the BDE or a similar engine needs to exist, but you don’t have to have it on the client side. In short, you now need only one set of server side database tools where before you needed database tools, on each client machine.
Distributed datasets are one means of cutting down on network traffic. After you have download data from the server, you can manipulate it on the client side, without initiating any more network traffic until you are ready to update the server. This means you can edit, insert, and delete multiple records without causing network traffic. When it comes time to update the server, you can send multiple data packets over the network at one specific, prechosen time.
Furthermore, using what is called the“briefcase model”, you can disconnect the client from the network and still access the data. Here’s how it works. You can:
- Save a remote dataset to disk, shut down your machine, boot up again, and edit your data without connecting to the network.
- You can then reconnect to the network and update the database. All of this occurs without the presence of large database tools on the client machine.
All of this adds up to the fact that you don’t have to actually be able to reach the server at all times in order to work with your data. This is ideal for laptop users, or sites where you want to keep database traffic to a minimum. There is a small (about 150 Kb) file called DBCLIENT.DLL that is needed on the client side, but that is very little when compared with the many Mbs of files required by the BDE or other database middleware.
Access to database constraints are another important aspect of the MIDAS technology. When you download data from the server, you can simultaneously download a set of constraints that will be automatically enforced. The constraints can help programmers ensure that the user enters only valid data. When you are reconnected to the network, your data can then be updated without mishap.
If perchance an error does occur while you are updating the dataset, then there are built-in mechanisms that can aid the programmer in reporting and handling the error. For instance, if a second user has updated a record that you are trying to update, then that fact can be surfaced, and the user can be given a choice of options on how to proceed. A prebuilt form that ships in the Delphi Object Repository makes it simple to implement error handling in your application.
Yet another important feature of Borland’s multi-tier computing includes distributing the load borne by a database over multiple servers, as well as providing fail-over capabilities in case of an error. This subject will be explored in more depth in the next section, and also in the technical sections found in the latter half of the paper.
Terminology
I want to spend a few paragraphs on terminology, since this is a new technology, and one that is being interpreted in various ways by different companies. This paper is about something called, at various times:
- distributed computing
- client/server computing
- multi-tier computing
- n-tiered computing
- remote datasets
- distributed datasets
and a host of other terms, many of which are often used out of context, or in a very loosely defined manner. The specifics of the definition of multi-tier computing used in this paper were outlined in the last section, entitled "An Overview of Borland’s Multi-tier Computing Technology." The definition is further expanded and refined in this section.
Multi-tiered computing is an industry-wide generic term for this kind of technology. In particular, multi-tier is the term for this technology that has been adopted by Borland.
Borland supports a three-tier technology, which in its classic form consists of:
- A database server on one machine,
- An application server on a second machine,
- And a thin client on a third machine.
The server would be a tool such as InterBase, Oracle, Sysbase, MS SQL server, etc. The application server and the thin client would be built in Delphi. In most scenarios, the database access software (e.g BDE, SQL*NET, etc.) would run on the same machine as the application server. Remember that this is simply the classic case, and there are many other configurations possible.
The Delphi team refers to their specific tools for implementing this technology as “distributed datasets”. In other words, Delphi implements a multi-tier technology via a set of components, and our docs refer to the technology supported by these components as distributed datasets.
Borland promotes this technology under the name MIDAS. Midas was a mythical king of Greece who received from Dionysus a gift enabling him to turn all he touched into gold. After a period in which all his food, and key members of his family, were turned into gold, he grew weary of the gift and was released from it by washing his hands in a river. The sands of that river were then turned into golden-colored sand, as visitors to Greece can still see today.
There are at least two distinct aspects to Borland’s distributed dataset technology:
- The built-in Delphi components that make this possible.
- The OLEnterprise product that provides extra support for distributed computing, and for load balancing.
Both the Delphi Components and OLEnterprise are included in MIDAS.
The built-in Delphi components make it easy for you to use DCOM to connect two machines and pass datasets back and forth between them.
The OLEnterprise tools enhance the core Delphi technology. In particular, the OLEnterprise tools provide an alternative to DCOM that simplifies the task of connecting two machines, and particularly of connecting two Windows 95 machines.
OLEnterprise comes from Borland’s recent purchase of the Open Environment Corporation. The purchase of OEC also gave Borland access to an Object Broker
that allows you to randomly distribute the load of a task across several servers. In particular, you can load your server tools up on several machines; then the broker will chose one of these machines each time you make a connection. For example, if you had 100 clients and three servers, then the Object Broker
would randomly divide the load across the three servers so that each had (approximately) 33 clients.
The broker also provides support for those occasions when a server is forced to shut down unexpectedly. By writing a few lines of code, you can provide fail-over services that would switch clients of a downed server over to a running server. Furthermore, the broker would never attempt to connect a new client to a server that has gone down, but would instead automatically connect them to one of the servers that is still running. I include a sample 20-line procedure later in this paper that demonstrates how to implement this fail-over process.
Technology Details: Using Distributed datasets
It is now time to start analyzing the technology involved in Delphi Distributed datasets. In other words, the paper now switches from theory to practice. This section gives an overview of the components involved in this technology, while the next section begins a detailed examination of the components.
To start, I will look only at the core Delphi components. Later in the paper I will look at the additional OLEnterprise tools that came from OEC.
There are four key Delphi tools that make distributed datasets possible. The first two appear on the server side:
- Remote data modules
are just like standard data modules, only they help you broadcast data not to your current application, but to locations on the network. In particular, they turn a simple data module into a COM object, thereby allowing you to access the data module from a remote server via DCOM.
- The TProvider component resides on remote data modules just as a TTable object can reside on a standard data module. The difference is that a TProvider broadcasts a table across the network. Provider objects are also included in the TTable and TQuery objects as properties. However, if you access them as standalone components, then you will have more flexibility and power. In particular, you hook up a TProvider component to a TTable or TQuery so that other programs on the network can access the data from the TTable or TQuery via DCOM. The job of the remote data module is to give clients access to the specific providers available on a server. The client first connects to the remote data module, then queries the remote data module for a list of available providers on its server.
On the client side, there are two components you use to access the data supplied by the server:
- The TRemoteServer component gives the client the ability to connect to the server, and particularly, to the remote data module on the server. More specifically, it connects to the COM interface supported by the remote data module. Despite the implication inherent in its name, TRemoteServers exist on the client side, not on the server side. TRemoteServer is the component that knows how to browse the registry in search of available servers. Once the server is found, the TRemoteServer will connect to it.
- The TClientDataSet component hooks up to the TRemoteServer component, and then attaches to a specific provider on the server. They give the data sources on the client app something to plug into when they want to connect to a remote dataset. In short, the TClientDataSet plays the same role as a TQuery or TTable, only it is serving up data from a remote site. Imagine the traditional TDatabase, TTable, TDataSource, TDBGrid configuration seen in many standard Delphi applications. In a remote dataset, you make a slight change to these configuration by using TRemoteServer, TClientDataSet, TDataSource and TDBGrid. In this new scenario, TRemoteServer plays a role roughly parallel to TDatabase, while TClientDataSet plays a role fairly similar to that traditionally played by TTable or TQuery. I don’t mean to imply a one-to-one correspondence between TDatabase and TRemoteServer, but a rough similarity can be seen between the roles played by the two components.
The diagram shown in Figure 1 depicts the architecture of a remote data set application. On the top of half of the picture is the server side of the equation, which consists of a remote data module, three tables, and three providers. On the client side, you find a TRemoteServer and three TClientDatasets. Attached to the client dataset you could have a series of data sources and visual controls. Notice that you need one provider and one client data set for each table you want to broadcast. Needless to say, you could have varying numbers of tables, as well as many other objects, forms, etc, on each side of the equation. My goal here is to focus on only the core elements in a proposed three-table scenario that must be present to make the concept of remote data sets work.
Figure 1
: The architecture of a remote dataset.
In this section,I will describe one possible technique for building a server for a remote dataset. Here is a quick overview of the steps involved:
- Start a new application, hook it up to some data as you would in any other Delphi application, and save it to disk.
- Use File | New to create a remote data module.
- Place one or more TProvider components on the remote data module.
- Hook the TProvider up to a TTable or TQuery object.
- Right click on the TProvider to create interface methods for accessing the provider on a remote machine.
- Save your work and run the server once to register it.
In this scenario, you can start by putting the TTable and related objects in a data module, and putting the providers in a remote data module. You should, however, in your final product, place the tables and the providers in the same remote data module.
I start with one data module and one remote data module simply because it gives you a familiar place to begin. It allows you to start with a well-known standard Delphi application, and then to isolate all the new code in one place; that is, in the remote data module. Ultimately, however, you should place both the TTable and TQuery objects directly on the remote data module, or else you might have synchronization problems.
Here is a more detailed example. To get started, construct a standard database application linking the Customer and Orders table from the DBDEMOS database. Put the tables on a data module, and just proceed as you would normally. On the client side, you will arrange these two tables into a one-to-many relationship, as shown in Figure 2. However, on the server side, you can simply place the tables on the form and data module and should not necessarily arrange them in a one-to-many.
It is important to note that in many cases you might not even bother to put a grid on the server side, since the server application is not meant for viewing or manipulating data. All you really need to do on an application server is put down some TTable and TQuery objects, and access them via remote data module. Remember that in your shipping version, you should place the tables and queries on remote data modules, not on a separate data module.
* * * Begin Note * * *
Note: You naturally have a choice as to how much logic you put on the middle tier; that is, the server part of your applications. For instance, it is your choice as to whether or not to arrange the tables in a one-to-many on the server, though in most cases you would not do this. If you do this, then when you query the server from the client, you will get only the records from the detail table that are currently visible on the server. This may, in fact, be exactly what you want, particularly if the detail table is large. However, if both the master and detail table are small, then you may prefer to access all records from both tables, which is best accomplished by leaving them in their raw form on the server, and hooking them together in a master detail relationship only on the client. Even if you do not connect them in a one to many on the server, you still have the ability to control how many records come over at one time, so there really is no particular advantage to setting up a one-to-many on the server.
Borland may place additional code on its Web site (www.borland.com), that will aid in placing more logic on applications servers, even when using the first version of MIDAS.
* * * End Note * * *
Figure 2: The interface for a simple application with a one-to-many between the Customers and Orders table from the DBDEMOS database.
After creating the basic Delphi database application, you can go to File | New and create a remote data module. You will be prompted to supply a class name. I would recommend giving this remote data module a descriptive name, such as CustOrdersRemoteData. Remember that the remote data module is just like a standard data module, except it has a COM interface on it.
Drop down two TProvider controls on the CustOrdersRemoteData module. These controls are on the same page of the Component Palette as TTable and TQuery. Select File | Use Unit to add the program’s data module to the uses clause of the remote data module. Now use the DataSet property of the TProvider objects to hook the providers to the TTable objects. An example program of this type, called CustOrdersServer, accompanies this paper. In your shipping version, however, you should have both your tables and your queries on the remote data module. Synchronization problems can occur if you have two remote data modules accessing the same tables on a standard data module.
* * * Begin Note * * *
Note: Remember to create sensible names for the components you use. For instance, I call the table that points at the Customer table CustomerTable. Its data source is called CustomerSource. The detail table is called OrdersTable and its data source is called OrdersSource. I name the provider that is hooked up to the Customer table the CustomerProvider, and so on.
* * * End Note * * *
It might be helpful to pause for a moment and see what we have created so far.
- A main form with two grids on it.
- A data module with the Customer and Orders table. You can optionally place the tables directly on the remote data module.
- A remote data module with two TProvider objects on it. One is connected to the CustomerTable and the other to the OrdersTable.
If you pause for a moment and look at the source for the remote data module, you will see that it contains an object that looks like this:
TCustOrdersRemoteData = class(TDataModule, ICustOrdersRemoteData)
CustomerProvider: TProvider;
OrdersProvider: TProvider;
private
{ Private declarations }
public
{ Public declarations }
end;
Class declarations of this type provide an implementation for a COM object. The COM object in question is called ICustOrdersRemoteData. You should note that ICustOrdersRemoteData is a dual interfaced COM object, which means it can be accessed from other applications via OLE automation, and from remote machines, via Distributed COM. ICustOrdersRemoteData has an accompanying type library, and is declared in a separate unit, as explained later in this section of the paper. (There is no reason to despair if you are entirely clueless regarding COM and OLE. It’s best if you understand this technology, but Delphi automates the process of using it to such a degree that you can proceed to use remote data modules with no understanding of COM or DCOM.)
As things stand now, you can access the TCustOrdersRemoteData object from a second application or second machine. However, there is no automatic way to access the two TProvider objects that appear on the remote data module. To give clients access to the providers, you need to add properties to the ICustOrdersRemoteData COM object.
To proceed from this point, all you need to do is right click on a TProvider object and select "Export XXX from data module" from the menu, where XXX is the name of your TProvider component. The code produced by this action is explained below. If you right click on the provider component and don’t see this option, then that means it has already been selected. In other words, the option is removed from the menu after you select it once. This same option is available from a TTable or TQuery object placed on a remote data module.
Alternatively, you can go to Edit | Add to Interface, and use this tool to add two properties to the COM object that is being created behind the scenes. If you want to get even more technical, you can perform this same act using the Type Library Editor. Of these three methods, the right click action is simplest to perform, so it is the default technique you should employ.)
If you select Edit | Add to Interface, a small dialog like the one shown in Figure 3 appears. Fill in the Declaration field with a line that says:
property CustomerDataSet: IProvider;
Needless to say, this adds a property called CustomerDataSet to the ICustOrderRemoteData COM object. The purpose of this property is to give remote clients access to the CustomerProvider object. The same end will be achieved if you simply right click on the object and select the appropriate menu item.
Figure 3
: Adding an interface to a COM object.
When you click on OK in the Add To Interface dialog, you will be taken automatically to a newly created method in the remote data module. This same method would be automatically created if you right clicked on the component and chose the appropriate option from the menu. The method itself should look like this:
function TCustOrdersRemoteData.Get_CustomerDataSet: IProvider;
begin
Result := CustomerProvider.Provider;
end;
The declaration, begin..end pair and single assignment statement for this method are created automatically by the IDE. If you used the Type Library editor to create the method you would need to write the singe line of code that returns the IProvider interface from the CustomerProvider object. In short, this task is taken care of for you automatically if you select Edit | Add to interface, or if you just right click on the component and select "Export XXX from data module" from the menu.
After adding the CustomerDataSet to the COM object, you should now add the OrdersDataSet by right clicking on the component. The method created by this action will look like this:
function TCustOrdersRemoteData.Get_OrdersDataSet: IProvider;
begin
Result := OrdersProvider.Provider;
end;
When using type libraries directly to create this method, you will notice that along with these Get methods, there is also a Set method produced for the property. You can leave it blank. In other words, it is not an error to have an empty method in your TCustOrdersRemoteData object that looks like this:
function TCustOrdersRemoteData.Set_OrdersDataSet: IProvider;
begin
end;
* * * Begin Note * * *
Note: In Delphi, by convention, all classes that begin with the letter I are interfaces; that is, they are COM objects. The CustOrdersServer application has a unit added to it automatically called CustOrdersServer_TLB, and it contains the declaration for the key COM object implemented by the TCustOrdersRemoteData class:
ICustOrdersRemoteData = interface(IDataBroker)
['{EE5A6C61-BD6E-11D0-A356-0080C751528B}']
function Get_CustomerDataSet: IProvider; safecall;
function Get_OrdersDataSet: IProvider; safecall;
property CustomerDataSet: IProvider read Get_CustomerDataSet;
property OrdersDataSet: IProvider read Get_OrdersDataSet;
end;
As you can see, this interface contains the two properties you just created on the TProvider components. In this case, the properties are called CustomerDataSet and OrdersDataSet. These are the properties accessed by the remote client via the remote data module. The other functions and procedures in the class are simply the get methods for these properties.
As explained in the Delphi documentation on COM objects, you never implement the methods of an interface directly. Instead, you use a standard Delphi class to implement them. In this particular case, the class in question is called TCustOrdersRemoteData, and it is implemented for you automatically in the remote data module unit.
At the same time that this COM object is being built up in the background of your application, a type library is also being constructed. You can access the type library by choosing View | Type Library from the menu.
*** End Note * * *
It turns out that the TTable and TQuery objects also have providers attached to them as standard properties. You could therefore leave out the two TProvider objects altogether, and write the following code:
function TCustOrdersRemoteData.Get_CustomerDataSet: IProvider;
begin
Result := DMod.CustomerTable.Provider;
end;
function TCustOrdersRemoteData.Get_OrdersDataSet: IProvider;
begin
Result := DMod.OrdersTable.Provider;
end;
You can also right click on a TTable or TQuery in order to select a menu option that lets you create the methods automatically.
The reason you might want to use the TProvider object is laid out in the online help as follows:“When you use a dataset component’s Provider property directly, Delphi creates and manages an IProvider interface for you behind the scenes. What you sacrifice is control. Using TProvider gives your application more direct control over the provider interface.” In particular, you should note the events associated with the TProvider component.
If you now save your work, and run the program once to register it with the system, then you will have created a remote data server. You can stick this object out on an NT server that has the BDE and some Delphi DLLs installed, and then access it from anywhere on the network if you are using OLEnterprise, or anywhere in your domain, if you are using DCOM. (Connection issues will be discussed in the last sections of this paper.) If you move it to a second machine, you should run it once on that second machine to register it with the system. When you access this server from a client, you only need to install the single Delphi client executable on the client side. No database tools are needed, other than the 151 Kb DBClient.dll file. (On the server side, you should include STDVCL32.DLL. This will be installed automatically if Delphi is on the server.)
To summarize the steps outlined in the section:
- Create a standard Delphi database application with the Customer and Orders table in it.
- Add a remote data module and drop two TProviders on it.
- Use the DataSet property of the TProviders to connect to the TTable objects used to access the Customer and Orders table.
- Right click on the TProviders to create methods that retrieve the appropriate provider interface.
- Compile the application and run it once on the client. You can now test your application. If you want to move the server to a remote machine, run it once on the client and once on the remote machine.
As you can see, it is simple to build an application server in Delphi 3.0. I have gone into such detail when explaining the process so that you can become familiar with the theory behind the process, thereby understanding not only how to build the server, but why it is architected in this particular manner.
MIDAS
