Data Explorer Tutorials

Data Explorer (DX) is a software package for the visualization of data. Put simply, it turns those thousands of numbers in a data set into an image thus allowing an easier understanding of the information.

NOTE: At the time of writing, this package was available within Dalhousie Physical Oceanography only on our IBM RISC System/6000 machines.

Although Data Explorer is an easy package to learn to use, tutorials have been created that let you quickly learn some of the basics. Although this tutorial was designed for users at Dalhousie Physical Oceanography, it was be useful to other DX users as well.

Videos on Creating Visual Programs and Data Importing and Module Writing are available. The IBM Visualization Data Explorer manuals (User's Guide, User's Reference, and Programmer's Reference) are also available.

Data Explorer has a extensive on-line help facility, which is basically a copy of the User's Reference. This can be accesses by pressing the middle mouse button on a module or any window.

Other sources of information on the Internet include:


The following is an index of the 3 tutorials.

Tutorial 1: Viewing Topology/Bathemetry Data


Tutorial 2: 3D Look, Vectors, and Animation


Tutorial 3: More Vectors and Adding Text



Tutorial 1: Viewing Topology/Bathemetry Data

This is the first in a set of tutorials on Data Explorer. In this example we show how to visualize elevation data. The data is set up on a regular grid, with the data representing the elevation at each grid point. Values above 0 represent land heights (topology), and values below 0 represent water depths (bathemetry).

We will build this visualization using modules, a pre-defined set of programs that manipulate data. When connected together, these modules form a program, or network diagram, that we will run to create an image.


Starting Data Explorer

To start Data Explorer, type "dx" at the command prompt.

NOTE: At the time of writing, Data Explorer is only available (within Dalhousie Physical Oceanography) on our IBM RISC System/6000 machines.

The Visual Program Editor window should appear on the screen. It will look like this:


FileSelector

The FileSelector module creates a window on the screen that is used to get a filename from the user.

To use the FileSelector module, go to the left of the Visual Program Editor window, and under Categories, select Interactor. Then below it, under Interactor Tools:, select FileSelector (as seen in the diagram below).

Move the mouse pointer to the right into the main work area. Position the mouse where you want to put the FileSelector module (usually at the top when beginning a program, you can change it later if necessary by clicking and holding the left mouse button on the module, and then moving it to the desired location), then click the left mouse button once. A small green box should appear (see diagram below), with the word "FileSelector" in it. This is called a module.

NOTE: There are two outputs from the FileSelector module. The left tab will output the filename with the complete pathname, and the right tab outputs only the filename. The left one will be used in this example, and is generally the most useful output.

To select a filename using the FileSelector module, double click the left mouse button on the module and a new window should appear (see diagram below).

Type in the filename "/data/dx/tutorial/data/elevation.general", or you can click on the "..." button, which will bring up another new window (see diagram below). In this new window you can select a file by clicking on the file list on the right. You can also change the pathname by clicking on the list on the left.

NOTE:


Import

The Import module is used to read data from the disk into Data Explorer. This is usually the first step of any visualization. The Import module is under the Import and Export category.

OPTIONS

The parameters of any module can be accessed by double-clicking on the module, or by going to the Edit menu and selecting Configuration.

The Name parameter is usually connected to the output tab of a File Selector module, to make selecting the file easier and less prone to typing errors. Alternatively you can specify the name of the file you wish to import here.

The format parameter is the type of file to import. This defaults to (file extension or content), which means that it will automatically change depending on the filename selected. It should be always be left as the default. Possible values for this field are "dx" (Data Explorer's own file format), "cdf" (NetCDF file format), and "general" (a general file format used with the Data Explorer "dx -prompter" command, see manual for further information on this).


Other Notes

"Tabs" are the little boxes on the top and bottom of the modules. The ones on the top represent input parameters, and the bottom ones represent outputs, which are used to connect to the inputs of other modules. If a tab is displayed outside of the module it is not set. If it is displayed inside the module than this parameter is already set. Also, light green colored tabs are parameters which must be set, while dark green tabs have default values which will be used if you do not specify values.

Tabs that are already set (i.e. the tab is pointing in towards the module, or the parameter is greyed out) can be unset simply by double clicking on the module to get the parameters of that module. Then click on the little box on the left beside the parameter name on unset the parameter.

Connect the left output tab of the FileSelector module to the Name parameter of the Import module. To accomplish this, click and hold the left mouse button on the bottom left tab of the FileSelector module. The name of the tab that you you are on will appear in the module box (it should read "FileSelector filename"). While still holding the left mouse button, drag the mouse to the leftmost input tab of Import. The name of the tab that you are going to connect to appears in the module box (it should read "name Import"). Now release the mouse, and a line should now be connecting the two modules.

The above diagram has the output tab of FileSelector connected to the name input of Import. Note the tabs that are set are inside of the module, and the tabs that are not set are outside of the module.

NOTE: Most of the other options of Import are automatically changed depending on the filename selected, and changing them from their defaults may cause an error. We will use the defaults in this tutorial.


Include

The Include module is used to select data with values within a certain range. It is under the Importing and Exporting category.

Connect the output tab of Import with the data input of the Include module.

OPTIONS

The min parameter sets the lower limit of data to be used. Any data value lower that this will be excluded.

The max parameter sets the upper limit of data to be used. Any data value higher that this will be excluded.

The exclude parameter is a flag that will reverse the way the module works and will exclude data values between min and max. This parameter is not set by default.

The cull parameter is a flag that if set to 0 will mark the points as invalid, and if set to 1 will remove the points. It is set to 1 by default. This will sometimes cause problems because it will make the grid irregular, so setting this flag to 0 is a good idea.


To set the parameters of any module, double-click the left mouse button on the module (the green box with the name of the module inside it).

Now set the parameters of the Include module. We will set min to -400, and set cull to 0, causing all points below -400 to be marked invalid. By not setting a max value, it will default to the maximum data value in the file.


Color

The Color module will color the data. It can color all data the same, or color the data different colors depending on the value. This module is under Transformation category.

Join the output tab of the Include module with the input tab of the Color module.

OPTIONS

The input parameter is the data to be colored. This is usually input from another module.

The Color parameter is the color to change the data. This can be input from the rgb output tab of the ColorMap module, or can be entered directly. Valid colors can be entered as a vector (i.e. [r g b], where r is red, g is green, and b is blue, between 0 and 1, like [0.5 0.5 0.5] which is a shade of grey, or [1 1 0] which is yellow). Colors can also be entered as "red", "black", "limegreen", etc.

The opacity parameter sets the translucency of the object. This parameter can also be input from the opacity output tab of the ColorMap module, or can be a value from 0 to 1. A value of 0 means that the object is completely transparent, and a value of 1 means that the object is opaque.

We will not alter the input parameters of color until the ColorMap module.


ColorMap

The ColorMap module lets you create colors using adjusting sliders. It also allows you to change the color of the data depending on its value. It is under the Special category.

Now connect the output tab of Include to the input of ColorMap. Include is already connected to the output tab of the Color module, but each module's output tab may be connected to several other modules.

Now connect the rgb output tab of ColorMap to the Color input of Color. Then connect the opacity output of ColorMap to the opacity input of Color. This will allow the ColorMap module to control the coloring of the data.

OPTIONS

The only input is the data parameter. This will automatically use the maximum and minimum values of the data set in the ColorMap Editor (see diagram below). To change the colors to be used, double click on the ColorMap module.

NOTE: The minimum and maximum values will only be automatically set after the program is executed and an image is formed. You might have to alter the colors after this first part of the tutorial has been completed to see these effects.

The default is to color the data values from red to blue, with red assigned to the highest data value and blue to the lowest.

To change the colors, you can move the anchor points (the little boxes on the line) in the Hue window. To create new anchor points, double click anywhere in the Hue window, and a new anchor will appear, along with the data value at that point on the side. The same can be done with the Saturation and Value windows. These add white and black respectively to the color. Experiment with these to get the desired color. The same can be done for the Opacity window.

The colormap for this tutorial might appear as follows:


Image

The Image module displays the image to the screen. It is under the Rendering category.

Image has only one parameter, the object input. Connect the output tab of Color to this tab. The output tabs of this module are hardly ever used.



Viewing Your Program's Output

You are now ready to see your data. Go to the top of the Visual Program Editor window, and under the Execute menu, select Execute Once. If you did everything correctly, after a few seconds of calculations, an image should pop-up on your screen. If not, then you will hear a beep and a message window will appear telling you where the error was, and the module with the error will appear in orange. Fix the problem and execute again.


The final Data Explorer program will look as follows:

The resulting image from the tutorial should appear as follows. Your results may vary depending on the colors you chose in the ColorMap Editor.


Tutorial 2: 3D Look, Vectors, and Animation

This is the second in a set of tutorials on Data Explorer. If you haven't looked at the previous tutorial, you might want to do so before starting this one.

In this tutorial we will augment the program from the first tutorial by making the image appear 3D and by adding vectors to the image. We will also add a color bar to the image, which will help to decipher what the colors on the image represent.


Adding a Color Bar

ColorBar

The ColorBar module will add a color bar to the image. This is useful for finding out what the colors in the image represent. It is found under the Annotation category.

OPTIONS

The ColorMap parameter is the colors to display in the color bar. This is usually input from the rgb output tab from ColorMap.

The position parameter tells us where in the Image Window to place the color bar. It is a vector with an x and y component, with values between 0 and 1. A value of [0 0] would put the color bar in the lower left corner, and [1 1] would put it in the upper right corner.

The shape parameter is the size that the color bar will appear in the Image Window. This is also a 2-vector, representing an x and y component in pixels.

The horizontal parameter is a flag that controls the direction of the color bar. The default of 0 would mean to make the color bar go down, and a value of 1 would mean to make it go across.

The label parameter just the title of the color bar. You can set this appropriately.


Connect the rgb output of ColorMap to the colormap input of ColorBar.

Double click on the ColorBar module, and set the following parameters:


Collect

The Collect module simply combines many objects into one object. It is under the Structuring category.

Collect's only parameter are objects. If you wish to add more objects, click on the Collect module, then go the Edit menu at the top and select Add Input Tab. This will create another input tab on the Collect module, allowing you to add another object to collect with the other inputs. This can be done to a maximum of 21 objects.

Join the output tab of ColorBar to the one of the input tabs of Collect, then do the same with Color and the other input tab of Collect. Disconnect the line joining Color and Image by clicking on the input tab of Image, dragging the line into empty space, and then releasing the mouse button. Now join the output tab of Collect to the input tab of Image.

Now Execute the program, and the new image should contain a color bar in the upper right corner of the image, as shown below:



Making a 2D Image into a 3D Image

RubberSheet

The RubberSheet module will create a 3D surface based on data values. It is under the Realization category.

OPTIONS

The data parameter is the data that will be used in the rubbersheet.

The scale parameter adjusts the scale factor of the deformation. If the image appears to deformed, lower the scale parameter. If the image appears to be a flat surface, increase the scale parameter.


Connect the output of Include to the data input tab of RubberSheet. Then connect the output of RubberSheet to Color.

Set the scale parameter to 0.30. Now Execute the program. You should not notice any difference. This is because you are looking at the image from above. To change the view from which you are looking at the image, go to the Options menu of the Image Window (make sure you do this from the Image Window, as the Option menu there are different from the Options menu in the Visual Program Editor window). Select View Control and the View Control window will appear:

Change the Mode to Rotate and go to the Image Window. You will now notice a co-ordinate plane with the X, Y, and Z axis labelled (Z comes out of the screen). You can click the left mouse button and move the mouse to change the orientation of the image. This will allow you to see the 3D look of the image, as shown here:

NOTE: Other Modes that might be useful are Zoom and Pan/Zoom. Using the left mouse button and selecting an outline will zoom in on that area, while using the right mouse button and doing the same thing will zoom out. The difference between Zoom and Pan/Zoom is that Zoom will always zoom in on the center of the Image window, and Pan/Zoom will let you choose the point on which to center the zoom.


Normals

The Normals module will compute normals on a surface, thus shading the image differently. This will make the image look more realistic. This module is under the Rendering category.

OPTIONS

The surface parameter is the object which you want to alter.

The method parameter alters the way the module adds normals. This defaults to "positions", which is what one would normally use.


Connect the output of RubberSheet with the surface input of Normals. Connect the output of Normals to the Color module.

Execute and note the change in the image, as shown here:



Adding Vectors

We will now add a set of vectors showing surface currents on the image. The vector data is arranged in a regular gridded form, and there are 50 frames in the data file. For more information on importing data, refer to the Data Explorer documentation.

New data must be imported for the surface current vectors. Select and then connect the following modules:

The added modules on the work area should look like this:


Select

The Select module chooses a member of a group or a list. In this case it will select a single frame from a group of frame data. This module is under the Structuring category.

OPTIONS

The input parameter is the list from which you want to choose a member.

The which parameter is the number of the member you want to choose. We will choose the default 0, to choose the first frame of the multi-frame series.


AutoGlyph

The AutoGlyph module transforms a vector or vectors into an arrow representation on the image. It is under the Annotation category.

OPTIONS

The data parameter is the vector(s) you wish to make into arrows.

The type parameter defines what kind of arrow to display. This is useful for making 2D vectors into 3D looking arrows, by specifying "rocket". Other useful values are "arrow" and "needle".Normally this is left as the default which will determine the appropriate type of arrow to display.

The shape parameter sets the thickness of the arrow. Set this to 1.4 for this example.

The scale parameter sets increase or decreases the length of the arrow.

The ratio parameter is the ratio between the lengths of the largest and the smallest arrows. If this value is set to 1, all arrows will be the same length. This value is normally kept at 0, and should be set at 0 for this example.


Connect the Include module to a Select module and set the which parameter to 0. Join this with an AutoGlyph module, and set these parameters: Choose a Color module and set the color parameter to Yellow. Connect the output of AutoGlyph to Color. Now connect Color to Collect (you will have to add an input tab by clicking in the Collect module, going to the Edit menu, and selecting Add Input Tab). The program should look like this:

Execute and a set of vectors showing surface current should be displayed on the image.

Now go to the Select module and change the which parameter to 1 and Execute. This will change the data to the second frame of data. This is not practical for viewing many frames of data, so we will use another module called Sequencer.



Animating Data

Sequencer

The Sequencer module generates a sequence of integers. It is under the Special category.

OPTIONS

The Sequencer modules has two parameters, min and max. These control the minimum and maximum values for the Sequencer to generate. If left as there defaults, they can also be set interactively by bringing up the Sequencer window (which can be done by double-clicking on the Sequencer module, or by choosing Sequencer under the Execute menu),

and then clicking on the "..." button.

Other buttons on the Sequencer window work like VCR buttons. Refer to the manual for a more in depth discussion of how these buttons work.


We will now attach the output of Sequencer to the which input of Select (if a value is already selected and the tab is not available, just change the parameter to a blank, and this will free up the tab).

Set the max of Sequencer to 49 and the min to 0. Next, bring up the Sequencer window as described above, then hit the "play" button (arrow pointing right). The vectors on the image should now change direction and magnitude. You can use the other buttons to play backward, pause, or stop.



Transmitters and Receivers

When the program grows large, it can sometimes become quite tangled and messy. To overcome this problem, we use Transmitters and Receivers. They allow connections without the use of visible connecting lines. Both of these modules can be found under the Special category.

To start, we will disconnect the lines connecting Color (the one attached to Normals) and Collect. Place a Transmitter module under Color, and a Receiver above Collect. These will have default names, but changing them will be much more descriptive and helpful in the future. Double-click on Transmitter and change the Notation field at the top to bathemetry. Any receivers that had the old name will automatically change to the new name. Now connect the Color module to the Transmitter module.

Do the same the outputs listed below and any necessary receivers:

The benefits of Transmitters and Receivers will be more obvious in Tutorial 3, when we add even more modules to this program.


The final program should look like this:



Tutorial 3: More Vectors and Adding Text

This is the last in a set of tutorials on Data Explorer. If you haven't looked at the previous tutorial, you might want to do so before starting this one.

In this tutorial we will add some different vectors and we will also add text to the screen that tells us the time of frame being displayed.


Adding a Wind Vector

We will now add a wind vector showing the direction and strength of the wind at Sable Island. There are also 50 frames of data in this file, corresponding with the frames of surface current vector data.

To add a wind vector, we do much the same as the vector field. Select and connect the following modules, and enter the options listed.

Now choose a Receiver and attach it to the Select module. Double-click on the Receiver module and change the notation to frame_num.

NOTE: only one Sequencer is allowed per program, therefore the same Sequencer must be used for parts of the program.

Choose and connect a Receiver to the Collect module. Change its notation parameter to wind (the same as the transmitter).

Execute the program again and now you should see a big red wind vector where Sable Island is located.



Sea Surface Height

To add a sea surface height indicators, we do much the same as the wind vector field. Select and connect the following modules, and enter the options listed. Most modules have more options available to them then what is shown. To see these extra paraters, double click on the module to get the parameters. Then click on the Expand button at the bottom. New parameters should now be present. Do this for the AutoGlyph module and set the min parameter to 40, and the max parameter to 150. This will scale the sea surface height indicators differently, making the differences between frames more noticable.

Do the same thing 3 more times using the filenames: coastal2.general, coastal3.general, and coastal4.general. Each of these files contains 50 frames of sea surface height data, each at a distinct shoreline location. Join the four sub-programs using a Collect module.

Now choose a Receiver and attach it to the which parameter of the Select module of each sub-program. Double-click on the Receiver module and change the notation to frame_num.

It should look like this:

Since all of the sub-programs each do the same thing with the data, we can delete three of the Color modules, three of the AutoGlyph modules, and three of the Tube modules. Connect the remaining modules to look like this:

Select a Transmitter module, name it coastals, and connect the output of Tube to its input (as shown above).

Select a Receiver module, name it coastals, and attach it to the Collect module that is attached to Image.

Execute the program again and you should now see the four green sea surface height indicators.



Adding Text Captions

Now we want to add a caption to the screen to show the number of the frame currently displayed.


Format

The Format module formats a string. It also allows you to add integer, floats, and strings from other modules to form a new string, much in the same way that the programming language C does. The Format module is under the Annotation category.

OPTIONS

The template parameter is the string that you wish to print. Anywhere that you want an integer to appear from one of the inputs, put a %d. Put a %f for a float, and a %s for a string.

The value parameter is the value to insert into a %d, %f, or %s. This parameter is optional.


Select a Format module and double-click on it to get to the parameters. For our purposes, we want the template parameter to say: "hour: %d". Select a Receiver module, name it frame_num, and connect it to the first value input of Format. Recall that from Tutorial 2 that frame_num is transmitted from the Sequencer module. This means that the caption on the screen will show the frame number in the caption.


Caption

The Caption module shows a caption on the image. It is found under the Annotation category.

OPTIONS

The string parameter in the string that you wish to display for a caption. This is sometimes the input from the Format module.

The position parameter works the same as the position parameter for the ColorBar module.


Choose a Caption module, and change the position parameter to [0.95 0.05]. Connect the output of Format to the input of Caption. Select a Transmitter, name it captions, and connect the output of Caption to the input of Transmitter.

Select a Receiver module, and name it captions, and attach it to the Collect module that is attached to Image.

Execute the program to see the final results of this data visualization program:



Extending Visualization Programs

Data Explorer (DX) is intended to allow the user to change the way data is visualized. Feel free to use this tutorial as a starting point, adding different features or change options as seems appropriate. The DX videos do a good job of explaining how to write your own modules, and the online documentation explains each module in sufficient detail. There are also several other on-line tutorials, as mentioned on the introductory page.