Sunday, 30 December 2007

Brazil NHCP

I have been too pre-occupied with my work to do some quality blogging; I have not written anything till now about my favorite topic – Electronic Health Records. This is first of serious of blogs on various EHR initiatives around the world.Most of the information that will be included is public information and my intention is to simplify the information about the implementation from what i understand while allowing the users also to examine the contrast in the patterns used by different implementations. I have been told that Brazil is one of the first countries to have a successful implementation of Nation Wide Electronic Health Records. The project was termed NHCP – National Health Card Project. This project began as early as 1999 and a clinical pilot began in 2001

Aim of NHCP

The project was started to create national patient identification and information system for the Brazilian Unique Healthcare System called SUS -- Sistema Único de Saúde. Its aim is to collect patient treatment information in a national repository of clinical data.

Architecture Principles and Components

Figure below shows the logical architecture of the project. NHCP System is divided into 5 hierarchical levels, from the bottom: Point of Care Level, Municipal Level, Concentrator Level, State Level and Federal Level. The top three levels of the system has Permanent IP based network and the lower two levels has Dial Up Network as shown in Figure below.

NHCP Project has five major components

Ø Telecommunication infrastructure
Ø Point of care terminals;
Ø Servers and database facilities
Ø Point of care terminal and servers software
Ø Security issues and national standards to represent transmit and store health information.

The architecture of the NHCP is centered on patient and users having health cards. The health cards adopted in the project are used only for IDENTIFICATION purposes of users and health professionals and not to store clinical data. Both the users’ and health professionals’ cards have embossed the unique identification number, name, date, city and state of residence. The point of care equipment specially developed for this project has the card reader. The access to the system is limited to health professionals authenticated by the card and password. All data captured at the point of care is stored in the servers at the municipal, state and federal level

The technologies that were used are based on Java technology and the XML data format. The Java framework is based on the following concepts

Ø The NHCP is XML message-based; all communication between machines is XML over HTTP.
Ø Every server node can send and receive XML messages to and from any another server node.
Ø XML messages are associated to Java business classes.
Ø All XML data is handled dynamically using Java Reflection.

All XML is validated according to a rigid specification (DTD, infrastructure, and domain), which provides the flexibility necessary to add or remove business classes, and to modify the XML validation rules. In addition, servers and clients must authenticate to a server node using digital certification. The figure below depicts a generic server node with its principal building blocks and data flows.

Standards and Data Sets

The transport protocol used is HTTPS (HTTP +SSL). The terminals of care and municipal servers communicate using XML. The specifications were not HL7 but specifications agreed between Ministry of Health and the providers contracted for the development of the project. Document Type Definition (DTD) has been used for specifications.

The essential encounter data set contains the following information

Ø Reason for the encounter
Ø Diagnosis
Ø Procedures Performed
Ø Patient’s work-out
Ø Medication
Ø Follow Up

The diagnosis information is entered using ICD-10

Informational Governance

The use of the database of the National Health Card aims to only the management of health services by the different spheres of government and was guaranteed not to be used for commercial purposes or for any reason which hinders with constitutional rights of the Brazilian citizens. The architecture incorporates a Information Governance policy covering five basic requirements: privacy, authenticity, integrity, access control and audit of health data tied to the National Health Card.

For authentication any addition or alteration of information in the system is linked to an operator duly registered both for operation of the server and the terminal. All the health professionals responsible for the entry of data to the system are authenticated using health professional card which contains a encrypted password on the card.For access control, the system allows the implementation of a policy of setting privileges of access to classes of operators and standards of disclosure. All attempts to access information and functions of the system are stored for auditing purposes.

From IG point of view the systems offline are treated the same way as that of online systems. The traffic between a terminal and a server, upload between servers or service is performed on the HTTPS protocol, or HTTP protocol (Hypertext Transfer Protocol) protocol on the SSL (Secure Sockets Layer).


Since spring 2003, the pilot infrastructure has been fully operational, consisting of:

Ø 8 million patient ID health cards
Ø 50,000 professional ID cards
Ø 3200 point-of care-clients
Ø 2 federal servers
Ø 27 state servers
Ø 13 concentrator servers
Ø 44 municipal servers

In parallel with the pilot project and in anticipation of consolidation for the rest of the country, 80 million people have been uniquely identified in the NHCP database, representing 40% of the Brazilian population. Interoperability with external systems is also operational. Systems from external vendors are communicating with the NHCP by exchanging XML documents.

NHCP server -side data:

Ø 180 different use-cases
Ø 1,800 Java classes
Ø 230,000 lines of code
Ø 700 different database tables
Ø 160 different XML message types
Ø 50 gigabytes of data/million users/year


NHCP allows data to be queries and profiled; for example diagram below shows actual data queried from the NHCP system for San Jose dos Campos, a city of 500,000 people near Sao Paulo. The x axis shows the treatment procedures codes, the y axis the date, and the vertical z axis shows the corresponding quantity. So what happened to cause this peak? From the data, it was possible to identify that the peak was due to the annual influenza immunization program; it was in winter, and over a two-week period, nearly 5000 people got their influenza shots.

My Observations

The project is simple implementation but strictly not a EHR implementation per-se and looks more of a data ware house implementation. There were claims that the message specifications were done after debate with clinicans who were the main users. It did not follow any reference model based specs so any of the changes to the specifications to keep in line with advances with clinical terminology and data will be adhoc. Also usage of ICD10 will limit the amount of data collected considering that its mostly disease based. The data sets indicate minimal information is captured and also a mechanism to retrieve from central repository to point of care systems is not well defined. The integration mechanism looks like a hardcore java implementation and non-implementation of a standard middle ware implementation will have its own technological refresh problems and scalability problems. Also some of the technologies used like DTD , XML binding has since become rarely used with the advent of XSD , SOAP , JAXB etc as the artcile on the SUN website indicates.

But the framework on the whole is a good start but building on the top of it may be bit difficult and any extensions to it may look out of place considering the usage of HL7V3.0 , SNOMED have become more wide-spread now.

Links, References, Acknowledgements and Copyright
Lemos M, Leao de Faria B. The Brazilian national health card project. Proceedings of the International Nursing Informatics Conference. Rio de Janeiro; June 2003. - Official Site with detailed info

Sunday, 2 December 2007

Understanding High Availability

This is a short article to define the concepts of High Availability to some one who is familair with Application Software but not with Infrastructure

High Availability is an option to create fault tolerant solutions to provide available and uninterrupted services to the users. High Available Solutions are developed by providing redundancy. Redundancy is provided by allowing a backup take over the role of the failed to avoid disruption of Service (Failover). Failover can be Stateless or Stateful. In a Stateless failover the solution providing the services is restarted without any restoration of data where as in a Stateful failover the data is check pointed to a standby hosting the solution offering the same services. High Availability needs to cater to both planned and unplanned types of service disruptions.

Availability is measured by the formula

Where MTBF is Mean Time between Failures. Calculated based on expected life of various components of a system

MTTR is Mean Time to Repair. Measurement of time required to bring a system back online after failure

High Availability is defined in terms of number of 9’s as shown in the following table
Number of 9’s Downtime / Year Typical Application

99.9% ~ 9 Hours Desk Top
99.99% ~ 1 Hour Enterprise Server
99.999% ~ 5 Minutes Carrier Class Switch
99.9999% ~ 30 Seconds Carrier Switch Equipment

Failover Design Principles

The basic design principles that were considered while evaluating the failover design are

Ø The failover design needs to be simple and transparent to the client who access the servers services
Ø The failover needs to be quick and the standby needs to be always booted up.
Ø There should be minimal manual intervention and the tasks involved in failover should be automated.
Ø The failover should ensure guaranteed data access, the receiving host should see exactly the same data as the original host.
Ø The failover design should not be designed only for current configuration but needs to consider future growth.
Ø The Operating System needs to be compatible with the failover design.

Failover designs are based on the concepts of Clustering, Load Balancing and Shared Storage.


Clustering, by definition, is to configure a secondary system as a standby for the primary system. If the primary system fails, the standby system automatically steps in with minimum disruption in service, takes over the functions of the original primary. The downtime due to a failure in the primary is limited to the takeover time.

A cluster requires various components to turn two systems / servers into a failover pair Servers. The components required are Network Connections (heartbeat network, service network, administrative network), Disks (Shared /Unshared), Application Portability and the basic design principle is there would be no Single Point of failure.

Load Balancing

Load balancing optimizes the performance of a system by distributing the traffic efficiently among a group of network servers which results in high availability and scalability of the system.

A single virtual IP address which maps to the addresses of each of the servers hosting the solution is reflected on the router-based load balancer. If a host is removed from and or becomes unavailable, the incoming request does not risk of hitting a “dead” server, since all host machines of the solution connected to the load balancer appear to have the same IP address.When a response is returned, the client sees it coming from the load balancer. In other words, the client is dealing with a single machine, the router based load balancer

Load balancers support different load-balancing techniques, which can be setup and configured
Ø Round robin. Assigns connections sequentially among severs in the cluster.
Ø Least connections. The server with the least number of connections gets the next connection.
Ø Weighted distribution. Divides the load among the servers based on individual weight or priority assigned to each server in the cluster. This technique is often used for the heterogeneous clusters that consist of servers running on different platforms.

Storage Area Network

Storage area network is a network for interconnecting storage and computing systems into a shared pool that many different hosts can access. SANs are based on Fiber Channel (FC) or SCSI over IP (iSCSI). SANs can be based on a number of different underlying technologies, including IP-based networks. The storage devices on the SAN are usually limited to disks and tape drives.

SAN is central for configurations, such as N+1, 2N or even the complex N-to-1, N-to-N, to failover in a smart, quick and efficient manner.

HL7V2.x and Low Layer Protocols

Minimal LLP (MLLP) and the Hybrid LLP (HLLP) are the two widely used lower layer protocols (LLP) for HL7 versions 2.x transmission. They are efficient and extremely simple to implement and use simple socket based communication. MLLP and HLLP does not really qualify as a "protocol" as they make use of TCP's reliability. MLLP and HLLP are simply a means of framing individual HL7 messages which is needed at the 7th level.

There are two types of LLP connectivity

Ø Persistent - In this type of connectivity socket is cached based on the endpoint and only one socket per endpoint is created and reused for the next set of messages.

Ø Temporary - In this type of connectivity a new socket is created for each message. When a message is sent the sender waits for the ACK and once it is received socket is closed.

The processing rules for both MLLP and HLLP are more of less the same; however, processing rules for HLLP include additional logic to signal transmission errors.

MLLP Wrapper

The MLLP wire format looks like <SB>dddd<EB><CR>

Where <SB> is Start Block, 1 Byte, ASCII <VT>, HEX <OxOB> and should not be confused with ASCII characters SOH or STX.

dddd is the HL7 variable length HL7 message. The data can contain any displayable ASCII characters and the carriage return character, . <CR>

<EB> is End Block, 1 byte, ASCII<FS> , HEX <Ox1C> and should not be confused with ASCII characters ETX or EOT.

<CR>is Carriage Return, 1 byte, ASCII , HEX <OxOD>

HLLP Wrapper

The HLLP wire format looks like <SB>tvv<CR>ddddCcccc<EB><CR>

Where <SB> is Start Block, 1 Byte, ASCII<VT> , HEX <OxOB>and should not be confused with ASCII characters SOH or STX.

t is Block Type, 1 Byte. ‘D’ is data block. This is used for HL7application message and HL7 ACK response message.‘N’ is NAK block and is used only as NAK to signal transmission errors

vv is Protocol Id, 2 Bytes. The characters ‘2’’1’ for this version

<CR>is Carriage Return, 1 byte, ASCII<CR> , HEX <OxOD>

dddd is is the HL7 variable length HL7 message. The data can contain any displayable ASCII characters and the carriage return character, <CR>.

In a NAK block, this field contains a 1-byte reason code as follows:
‘C’ is character count wrong in previous data block received.
‘X’ is checksum wrong in previous data block received.
‘B’ is data too long for input buffer in previous block received.
‘G’ is error not covered elsewhere.

Ccccc is Block Size, 5 bytes and is the character count of all characters so far in the data block up to and including the last data character. For this version of the protocol this is 5 + the size of the dddd field. An encoded HL7 message ends with a <CR>character. This character is considered part of the data.

Xxx is Checksum, 3 bytes an X-OR checksum of all character in the block up to and including the last data character. The checksum is expressed as a decimal number in three ASCII digits. If the value is 999, the checksum should not be computed. Processing will proceed as if the checksum were correct. This feature is used for applications where the messages will be translated from one character set to another during transmission.

<EB>is End Block, 1 byte, ASCII<FS> , HEX <Ox1C> and should not be confused with ASCII characters ETX or EOT.

<CR>is Carriage Return, 1 byte, ASCII , HEX <OxOD>