Sixth Quarter PSC Visible Human Subcontract Status Report

1) Description of progress towards completion of quarterly milestones & deliverables:

    Primary areas of progress during the quarter include construction and display of surface mesh models, extension of the platform independent Volume Browser (VB), compressed volume delivery from the volume server, and continuing improvements to network connectivity. Our work on production of surface meshes for segmented organs and tissues is greatly enhanced by the addition of new staff member Demian Nave. Demian's areas of interest and expertise include mesh construction methods and parallel computation. Given that we have access to, and make use of consulting from additional PSC staff and student aids, we do not anticipate further staff changes for the duration of the project. Using the program "powercrust" by Nina Amenta for surface mesh generation we have been able to use all of the hand segmentations produced by the Michigan team to produce first level meshes for the dozen objects they had segmented. Some of these have arrived in the form of a compacted run length coding, which we were able to use with no difficulty, while others were delivered as a variation of photoshop layer files. We have revised the layer extraction process written earlier in the project to handle both photoshop formats. Additional computed segmentations at PSC produced a full resolution skin surface mesh useful as an enclosing context model. Powercrust implements a provably good algorithm for generating approximating surfaces from dense point sets. Most of the current hand-segmented organs are not sufficiently dense in the vertical direction, so some of the models produced from this input have regions of incorrectly placed surfaces and related artifacts. Problems also arise from manual contours which are not oriented relatively normal to the local surface. These problems will be eliminated by 3D computational segmentation for areas between manual guide contours coupled with the ability to rapidly produce contours at arbitrary well chosen orientations by means of VB segmentation features. Even with these temporary restrictions, the end results are found to be quite useful as navigational aids in both VB and EW viewers.
      Raw surfaces generated by the existing process and its planned enhancements tend to be large because the meshing algorithm inserts additional points at intermediate positions to define the approximating surfaces. The number of triangles used in practice generally has to be reduced before viewing for efficiency of display and good user response time.
Because the great detail in completed models such as the skin exceeds visible resolution more usable simplified meshes were generated using Michael Garland's "qslim" program, which was highly recommended by Murial Ross. Qslim transforms the fine surface mesh composed of vertices and triangles into a topologically consistent but coarser mesh containing a user-specified number of triangles. The program was used to simplify several of our surface meshes. The largest model, the skin, which had already been partially simplified during production, contained 381,138 vertices and 762,000 triangles. For use as a navigational context surface this was successfully reduced to 7,500 triangles to allow rapid transmission and display with no significant change from the topology of the original surface.
     During the quarter we have looked at additional tools for Visible Human segmentation from outside the UMICH NGI VH project. Work in the Segmentation and Registration Toolkit (SRT) is progressing and should eventually be able to handle a number of the processes we need. Bill Lorensen has indicated that the SRT may be available in alpha release for our group in September but that it will require a significant learning period. We also evaluated the IsoView program from John Stewart. This program can produce some very nice results given enough effort, but can be very slow in operation and is limited in the size of the data volume that can be manipulated at any one time.
        Because restrictions in using these alternate tools do not fit all of the time constraints of our project we have been adding segmentation capabilities into the VB framework. We consulted with Brad Smith and other members of the Michigan team to determine key areas where a combination of manual input and computational aids would produce the greatest benefits toward rapid 3D model construction. Our resulting solution is implemented as an extension to the current Volume Browser, which lets us take advantage of its existing code and the networked interface to work with an entire visible human dataset with no restrictions on memory or region size. The primary segmentation facility is based on Catmul-Rom spline segments with movable control points to let anatomists sketch contours in any planar orientation. For best results anatomy markup should be done in an orientation that is close to perpendicular to the anatomical surfaces being outlined. During processing a spline simplification process is used to minimize the number of control points needed to maintain an accurate description of the contour.
      Rather than force users to precisely paint contours to voxel precision we provide computational aids to reposition contour control points based on template matching from a very small number (typically 1 to 4) of carefully positioned points which represent characteristics of the local surface. One of these template points should be provided for each type of surface interface such as bone to fat or fat to muscle etc. Our implementation applies each template over the context to which it applies and tries to avoid using a template on an inappropriate area. Contour refinement within the viewing plane is instantaneous from the human time scale. Users can also copy contours from layer to layer and use the previous templates to quickly adjust each new contour according to the (usually small) changes from slice to slice. When the user sees leaks or other aspects of the contour that they do not agree with the have the option to redraw any portion of the contour as a manual override or select new template points as the working context changes.
We have now provided a means for the VB client, functioning in this case as an anatomy labeling workstation; to send completed contour data back to the ES-40 server where we will use the additional computational power to fill regions between contours using the manually produced contours as both geometric guides and a source of statistics on voxel characteristics inside and outside of the contour. The fill in process that we are producing generates a complete 3D voxel level point cloud to define the surface and hence closes all gaps left from the users manual contours. These dense point sets fit all of the criteria for surface generation by powercrust. Models from this process will be inserted into the web accessible model collection so they are available by the anatomist and other users to view and revise almost immediately.
       Additional features of VB, including a measurement ruler and color space display, are described in a user manual and from the web site, http://vhserv.psc.edu:8000/volume_browser/index.html, we have set up to make the program available to the rest of the project team and other members of the VH community. A major benefit of VB is that it now supports all major platforms including UNIX, Linux, Windows 95/98/ME/NT/2000 and Mac OS X. Additional support for Mac OS 9 and BeOS can be produced if there is user interest. Operating as a browser VB accesses the same ES-40 volume server at PSC that is also used to feed Edgewarp. VB provides the essential framework for us to quickly test implementation methods and produce code modules that will be released to the rest of the project.
       Important features which have been demonstrated at the quarterly meeting, the NREN workshop and at Stanford include interactive arbitrary orientation slice browsing, display of shaded surface models, use of a 3D context window to supplement slice views, linkage to web browsers, linkage to Alex Ade's database at Michigan and use of a new 3D point identification service from PSC. Michael Gill from the NLM was able to be at both the NREN and Stanford demonstrations and expressed interest in arranging a large multi-user demonstration, which can be accomplished with the current level of system development. During the Stanford demo we were also able to show concurrent operation by Steve Senger from his facilities at Wisconsin. Parvati Dev plans to use both VB and Edgewarp with the NLM VF data and Stanford's Lucy2.

      The new point identification server at PSC uses data from segmented models to produce a translation from XYZ position to the nearest enclosing segmentation represented as a sequential model number. The model number then allows reference to an http based web server and to the anatomy database at Michigan. The database provides additional information about further levels of anatomical hierarchy and descriptive text. Since 3D surface models were being supplied from the XP1000 http web server at PSC portions of the recent demonstrations accessed two different servers at PSC along with the database server at Michigan. VB does not include the morpohometric features of Edgewarp but to the extent possible we have agreed to share common descriptions of orientation, position and scale so that each of these tools can be used to best advantage as needed.
   These recent demonstrations included operation from the compressed service port coming from the ES-40 at PSC. We are currently running the original uncompressed data service along with a relatively simple block of voxels compression and the more complex hierarchical compression. These services run on ports 8694, 8695 and 8696 of the ES-40 and allow us to test and tune proper compression levels according to user feedback and types of use. It is apparent that one level of compression does not simultaneously optimize users on low speed and high speed links. Anatomy users have also shown a strong preference to work mostly in the 3 primary planes rather than at arbitrary orientations. Consequently, we will be further optimizing compression and server modes to optimize those views. Tests of compression show visually lossless operation at 10:1 compression ratio, but much higher ratios are usable with slightly reduced quality and on platforms with fast processors to allow for the more demanding decompression codes. Lossless overlay at the final level operates at a 3:1 compression ratio limited by the data SNR.
   We have continued to make improvements to the delivered network performance of the volume server based on PSC network upgrades, software tuning and network scrubbing with the help of Web100 tools, enhancements to the server programs and operating system upgrades. The most important improvement to Web100, which has let us uncover bottlenecks, is the support for SMP configurations such as our 4 processor ES-40. The server was initially constrained to operation from a single processor but now uses all 4 according to system usage. Web100 modifications were also made to prepare for the next level of Linux upgrade, which further improves SMP support. A paper being prepared by Tom Hacker at Michigan outlines the use of Web100 in conjunction with the volume server to tune the network and application software.

2) Problems encountered during this quarter:

   Other than the time difficulties involved in juggling priorities to make even and rapid progress on all of the technical fronts given the budgeted staff level, there were very few significant problems during the quarter. Most of the problems that did occur have already been mentioned in the earlier description of progress, such as new Photoshop layers descriptions, limitations due to spacing of manual contours, and powercrust limitations. It has become clear that the current widget set used in VB needs to be replaced or at least supplemented to handle some of the requests from the UIT group and the demands of the segmentation and labeling role. The only other difficulty was scheduling of PSC network upgrades associate with the Terascale Computing System (TCS) installation.

3) Resolution of problems:

     Resolution of most problems has been described earlier. The remaining problems with surface construction will be greatly reduced by 3D point cloud completion and mesh generation that is a major part of the work continuing into the new quarter. Problems with network structure upgrades were resolved by cooperation of additional network and support staff beyond those already on VH project. This cooperation provided a temporary work around network path that was used in preparation for and during the NREN demonstrations at NASA. Permanent network upgrades taking place during July and August and will remove the need for these short-term measures and will be the final PSC network configuration provided for the current project. To assure reliable operation during recent demos and also for the increasing usage of the volume server we have added a third power supply so we can now continue to operate through any single power supply failure. (Tests from the TCS project show this as the single most likely hardware failure.) Additional tuning of the user interface controls for VB and compression levels will be done based on feedback from the user interface team (UIT).

4) Goals for the next quarter & action plan for their completion:

Many of the goals for the next quarter have already been discussed as continuations of work from the previous quarter. Increasing portions of the new work will be based on feedback from the UIT test results. UIT feedback will be most useful to adapt the underlying technical structure that has been developed to the educational mission of the anatomy lab.
However, independent of UIT results, it is clear that the largest work item continues to be anatomy segmentation and labeling. Our work in this area includes the anatomy labeling workstation extensions to VB, as well as 3D surface completion and meshing at the ES-40 server. Production of skeletal models is a high priority and we will provide service of the CT data to support that work during July. Work is underway to provide rapid user feedback from initial manual segmentations by transmission to the ES-40 server with model sign-out by simultaneous users operating in parallel from PC and Mac clients. Partial replacement of the VB widget set will streamline user operations during segmentation and labeling as well as in the educational user interface. Besides the template-matching mode we will be supporting additional modes for user directed segmentation using textures and colorspace markups.
Although we have demonstrated initial linkage of web browsers, VB, Edgewarp, the point identification service and the anatomy database, there is a great deal of additional work during the next quarter to enhance and streamline that linkage and to provide uniform modes of operation. That linkage is the foundation of flexible label display that will be shown during the new quarter. Additional areas of work include the export of VB components for use in Edgewarp and to support Java based interfaces, zooming in the VB context window, support for NURBS surface descriptions in the form of OBJ files, more interaction with the Stanford group, implementation of collaborative operation features, color and contrast transforms to approximate the 70mm film response and general performance improvements from code optimizations and special treatment of primary viewing orientations.
An operating system upgrade to the Linux 2.4 kernel will be done on both the ES-40 and XP-1000 servers. Besides general bug fixes and performance improvements the most important aspects of the upgrade are the significantly improved multiprocessor support, better virtual memory implementation and improved multi-threaded network stack. This will translate directly into better support for more users at higher rates of network traffic, which will let us take advantage of the network upgrades coming from the TCS project.

5) Next quarter needs:

In addition to developing new features we need to be sure to allocate enough time for optimization and completion of debugging of existing features that have been demonstrated in prototype but are not yet in finished form. We will be purchasing replacements for borrowed network cards that have been used during development and demonstrations as well as Nvidia based PCs for continued development and testing.
------------------------

_____
Arthur W. Wetzel awetzel@psc.edu Phone: 412-268-3912 Fax: 412-268-5832
Pittsburgh Supercomputing Center 216A Mellon Institute http://www.psc.edu
4400 Fifth Avenue Alt Fax: 268-1490(bus office), 268-8200(biomed)
Pittsburgh, PA 15213 http://www.psc.edu/~awetzel/