Bondage | Going to hell in a handbasket?

Notification Caption x
	Notification box

Notification Caption

Notification box

Author	Topic
WayaAzgitzka	Posted: 2009-07-03 12:25
Forum Regular United States 798 Posts	iPhone overheating. SMS issues with them as well. Now this.... London Stock Exchange to abandon failed Windows platform "Anyone who was ever fool enough to believe that Microsoft software was good enough to be used for a mission-critical operation had their face slapped this September when the LSE (London Stock Exchange)'s Windows-based TradElect system brought the market to a standstill for almost an entire day. While the LSE denied that the collapse was TradElect's fault, they also refused to explain what the problem really wa. Sources at the LSE tell me to this day that the problem was with TradElect." http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform -- "We are all faced with a series of great opportunities brilliantly disguised as impossible situations." - Charles R. Swindoll

pensive	Posted: 2009-07-04 08:50
Forum Maniac United States 5,860 Posts	the way we do it here is to have a new development system (in its entirity) running and tested in parallel with actual user loads and databases BEFORE the new system is put into use. and this assumes the testing was for a sufficient duration and worked properly under what software and hardware stress would normally be expected in typical and peak operations. sounds to me they were using a system which was being actively developed + debugged at the same time it was being used. that's a no no here. the press release indicated the system included somewhat older parts of MS backroom gear such as SQL 2003 and Windows Server 2000. '...TradElect runs on HP ProLiant servers running, in turn, Windows Server 2003. The TradElect software itself is a custom blend of C# and .NET programs, which was created by Microsoft and Accenture, the global consulting firm. On the back-end, it relied on Microsoft SQL Server 2000...' ==================================================== http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform ==================================================== in addition, a system like this should have had something HA-like in the background, and again, something tested BEFORE being set on stage for prime time. we use both Windows Server - SQL and Red Hat LINUX - SQL here. we have had our share of issues with both platforms. they aren't the sort of things you can set and forget. even after following what we thought were the proper testing methods, we have had unexpected surprises.

CloseToon	Posted: 2009-07-04 09:05
Evil On Board United States 16,351 Posts	pensive: the way we do it here is to have a new development system (in its entirity) running and tested in parallel with actual user loads and databases BEFORE the new system is put into use. and this assumes the testing was for a sufficient duration and worked properly under what software and hardware stress would normally be expected in typical and peak operations. sounds to me they were using a system which was being actively developed + debugged at the same time it was being used. that's a no no here. the press release indicated the system included somewhat older parts of MS backroom gear such as SQL 2003 and Windows Server 2000. '...TradElect runs on HP ProLiant servers running, in turn, Windows Server 2003. The TradElect software itself is a custom blend of C# and .NET programs, which was created by Microsoft and Accenture, the global consulting firm. On the back-end, it relied on Microsoft SQL Server 2000...' ==================================================== http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform ==================================================== in addition, a system like this should have had something HA-like in the background, and again, something tested BEFORE being set on stage for prime time. we use both Windows Server - SQL and Red Hat LINUX - SQL here. we have had our share of issues with both platforms. they aren't the sort of things you can set and forget. even after following what we thought were the proper testing methods, we have had unexpected surprises. Granted your perspective is right on, for proper development technique. However, testing a system like this isn't quite as practical as one might think. Even if you replay live trading into it, while tracking data via the Windows Monitoring Interface (WMI) and the SNMP data from the network, the reality is this kind of system isn't like your average web application. These systems demand continuous, live, unrelenting transactions. There's extremely limited room for error, even at gigabit ethernet speeds. Worse most of these vendors list their 'performance' based on burst speeds. Pound on these systems hard enough and long enough and all kinds of nasty surprises will surface. In many systems like this they resort to multicasting or UDP because they rapidly discover the compounding losses in the 'hardware'. They also rapidly discover that most NICs and other hardware aren't equal. Even with multicasting, surprises in the underlying OS will start to crop up (been here...done this.) I seriously doubt (from experience with trading systems) that even a hard tested system would have turned up some of these problems. Worse on scale once you start to find problems, your cost rapidly grow. Toon One serious drawback to the Windows platform is that if you do find a low level problem you are out of the repair loop. I have personally helped code special communications drivers at just above the hardware level and having a great big black hole (closed source software) between you and the application(s) is just suicide. Edited by - CloseToon on 2009-07-04 09:08:38

Author

Topic

WayaAzgitzka

Posted: 2009-07-03 12:25

Forum Regular

United States
798 Posts

iPhone overheating. SMS issues with them as well. Now this....

London Stock Exchange to abandon failed Windows platform

"Anyone who was ever fool enough to believe that Microsoft software was good enough to be used for a mission-critical operation had their face slapped this September when the LSE (London Stock Exchange)'s Windows-based TradElect system brought the market to a standstill for almost an entire day. While the LSE denied that the collapse was TradElect's fault, they also refused to explain what the problem really wa. Sources at the LSE tell me to this day that the problem was with TradElect."

http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform
--

"We are all faced with a series of great opportunities brilliantly disguised as impossible situations."
- Charles R. Swindoll

pensive

Posted: 2009-07-04 08:50

Forum Maniac

United States
5,860 Posts

the way we do it here is to have a new development
system (in its entirity) running and tested in
parallel with actual user loads and databases
BEFORE the new system is put into use.
and this assumes the testing was for a sufficient
duration and worked properly under what software and
hardware stress would normally be expected in
typical and peak operations.

sounds to me they were using a system
which was being actively developed + debugged at the same time
it was being used. that's a no no here.

the press release indicated the system included
somewhat older parts of MS backroom gear such
as SQL 2003 and Windows Server 2000.

'...TradElect runs on HP ProLiant servers running, in turn, Windows Server 2003. The TradElect software itself is a custom blend of C# and .NET programs, which was created by Microsoft and Accenture, the global consulting firm. On the back-end, it relied on Microsoft SQL Server 2000...'

====================================================
http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform
====================================================

in addition, a system like this should have had
something HA-like in the background, and again,
something tested BEFORE being set on stage for
prime time.

we use both Windows Server - SQL and Red Hat LINUX - SQL here.
we have had our share of issues with both platforms.
they aren't the sort of things you can set and forget.
even after following what we thought were the
proper testing methods, we have had unexpected surprises.

CloseToon

Posted: 2009-07-04 09:05

Evil On Board

United States
16,351 Posts

pensive:
the way we do it here is to have a new development
system (in its entirity) running and tested in
parallel with actual user loads and databases
BEFORE the new system is put into use.
and this assumes the testing was for a sufficient
duration and worked properly under what software and
hardware stress would normally be expected in
typical and peak operations.

sounds to me they were using a system
which was being actively developed + debugged at the same time
it was being used. that's a no no here.

the press release indicated the system included
somewhat older parts of MS backroom gear such
as SQL 2003 and Windows Server 2000.

====================================================
http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform
====================================================

in addition, a system like this should have had
something HA-like in the background, and again,
something tested BEFORE being set on stage for
prime time.

Granted your perspective is right on, for proper development technique.

However, testing a system like this isn't quite as practical as one might think.

Even if you replay live trading into it, while tracking data via the Windows Monitoring Interface (WMI) and the SNMP data from the network, the reality is this kind of system isn't like your average web application.

These systems demand continuous, live, unrelenting transactions. There's extremely limited room for error, even at gigabit ethernet speeds. Worse most of these vendors list their 'performance' based on burst speeds.

Pound on these systems hard enough and long enough and all kinds of nasty surprises will surface.

In many systems like this they resort to multicasting or UDP because they rapidly discover the compounding losses in the 'hardware'. They also rapidly discover that most NICs and other hardware aren't equal. Even with multicasting, surprises in the underlying OS will start to crop up (been here...done this.)

I seriously doubt (from experience with trading systems) that even a hard tested system would have turned up some of these problems. Worse on scale once you start to find problems, your cost rapidly grow.

Toon

One serious drawback to the Windows platform is that if you do find a low level problem you are out of the repair loop. I have personally helped code special communications drivers at just above the hardware level and having a great big black hole (closed source software) between you and the application(s) is just suicide.

Edited by - CloseToon on 2009-07-04 09:08:38

Bondage Forums

Forum Home | Active Topics | Subscriptions | Search | FAQ | AUP