Vail application solutions run on Vail's fault-tolerant, distributed voice platform that extends across two redundant sites in Chicago, Illinois and Deerfield, Illinois. One data center is collocated with Level(3) Communications in a carrier-grade telephony suite. Our second site is located at our corporate headquarters. Both locations are protected by full power backup, environmental control systems including chemical fire suppression, and restricted personnel access.

Each interconnected site includes a number of Session Initiation Protocol (SIP)-based Voice-over-IP (VoIP) voice clusters. Individual servers in each voice cluster are Pentium class PCs running various services — including automated speech recognition (ASR), text-to-speech (TTS), call conferencing, messaging (voicemail, email, FAX, paging), call analysis (DNIS, ANI, etc.), and web services — without employing any traditional proprietary telephone or switching equipment. Each site also includes at least two fully replicated RAID 5-based database servers. When additional service capacity is necessary, adding a new server is a plug and play operation due to the platform's modular architecture.

View an interactive diagram of the Vail Voice Platform (requires Flash)

Both data centers described above operate in tandem with both sites live on every customer application.

Each site is connected to multiple long-distance and local provider switches via multiple DS3-level access points on a SONET Ring. In our Chicago data center, there are redundant telephony access points, which protect against a local fiber cut. In the event that a carrier experiences an outage, all traffic can be moved to another carrier without interruption.

For Internet connectivity, Vail also runs multiple data service providers in tandem. These providers are fully redundant. A DS3 WAN connection links both data centers. If Internet connectivity is lost at either site, traffic is still routed via our WAN connections.

There is no single point of failure within the network. Our Deerfield site has maintained a 99.95% up-time for the last 5 years, and the Chicago site has not been down since going into production in May, 1999. The platform has processed tens of millions of mission-critical phone calls across over 50 unique applications and 13,000 toll-free numbers. On average, the platform operates at less than 15% of total capacity. This allows Vail to handle spikes in call volume seamlessly. If the allocated telephony capacity is exceeded during a spike, calls can overflow to other Vail trunk groups provisioned to service all applications.

Vail understands the many factors that threaten the reliability of voice applications today. From system hardware and sub-system interoperability, to telecom service provisioning, to the complexities of today's speech recognition software, applications need round-the-clock care to guarantee maximum uptime and uninterrupted service.

Vail's 24/7 Network Operations Center (VNOC) provides monitoring and testing services ranging from user-side application testing to in-depth port-level monitoring, tracking and reporting. Monitoring screens at our operations center provide a graphical and textual description of every critical element of the Vail Voice Platform and all live applications. Trained VNOC technical personnel monitor every element in the Vail Voice Platform, respond to resource- and port-specific alarms, and regularly check telecom and data service availability. As required, VNOC staff also make regularly scheduled test calls to client applications to ensure application-level functionality.

All servers on the Vail Voice Platform are monitored. Each machine has a redundant partner. In the event of a machine failure, a redundant partner server will handle the load of the failed machine until it is brought back online.

At the application level, Vail provides for a "heartbeat" to and from the host computer. This assures not only that the host computer is reachable, but also that the required application running on a given server is providing service. In the event that the Vail Voice Platform detects a problem with a host computer's service, the system automatically cuts over to a backup host computer.

Each platform service is also responsible for reporting access failures. In non-critical cases this data is written to a log file for later review. For critical host services the event is reported to a central monitoring facility in addition to being logged.

The VNOC also tracks application and platform behavior patterns over time to ensure future application reliability and security. Analyzing historical data often reveals performance inconsistencies (e.g. call duration, call volume, etc.), unusual call activity that may be linked to unauthorized use, and spikes in system resource consumption. The VNOC can suggest application optimization and tuning methods and recommend necessary steps to prevent critical failures before they occur.

The practices mentioned above are conducted according to documented guidelines and comply with customer security requirements. These practices allow Vail to resolve platform and application issues without undue delay or finger pointing, and ensures that callers can continue to access the system without any degration in caller experience.