Sybase Blogs

SQL/RPR available for initial public review

Glenn Paulley, Director, Engineering — Fri, 11 May 2012 16:36:36 EST

SQL/RPR (Row Pattern Matching), a proposed new part to the ISO SQL standard under development in the United States under ANSI is now available for initial public review and comment.

Row-pattern matching in the standard is something I have blogged about previously, and has seen some other initiatives in the academic literature as well.

SQL/RPR is all about complex analysis of data streams, though in a SQL context (ie. over persistent data) rather than over a streaming database system. The big deal is supporting predicates whose expressions can refer to different tuples of an ordered intermediate result simultaneously.

Here's the overview from RPR from the INCITS website:

This standard will specify the syntax and semantics of a new SQL capability to perform complex queries involving the relationships between many rows in a single (virtual or base) table. Detection and use of such relationships are critical aspects of many high-value applications. Sometimes called complex event processing, many business processes are driven from sequences of events. For example, security applications require the ability to detect unusual behavior definable with regular expressions. Financial applications that detect stock patterns are widely demanded. Fraud detection applications must recognize patterns in financial and other transactions. RFID processing requires the ability to recognize valid paths for RFID tags.
Extremely high interest in these capabilities has been shown by financial institutions, by the USA Department of Homeland Security and other government agencies, by large retailers and their suppliers, and transportation companies, among others.
Upon approval of the national SQL/RPR standard, which will be written as an amendment to Part 2 Foundation [FoundFDIS] of [SQL 2008], H2 [sic; should be DM32.2] expects to submit it to ISO/IEC JTC 1 possibly using the Fast-Track process with proposed maintenance in JTC 1/SC32 if approved.

The public review period is from now until June 18, 2012. The draft document is available here currently for a price of US$30. The $30 fee appears to be in error as the intent of the public review was to make the draft of RPR freely available. I hope to have an update on that in the very near future.

N. Carolina PowerBuilder User Group Meeting

John Strano, PowerBuilder Evangelist — Wed, 02 May 2012 07:00:15 EST

Thursday, May 10th 6:00pm - 8:00pm

Location: 10735 David Taylor Drive, Suite 100; Charlotte, NC 28262

Topics include an overview of PowerGen build tool and using .NET visual controls in PowerBuilder Classic.

*** The meeting will be available as a Microsoft Livemeeting session for any virtual attendees. Your registration confirmation email will contain the LiveMeeting link.

Call for participation: DBTEST 2012

Glenn Paulley, Director, Engineering — Mon, 16 Apr 2012 10:51:58 EST

The Fifth International Workshop on Testing Database Systems will be held on 21 May 2012, co-located with the ACM SIGMOD Conference held in Scottsdale, Arizona. The aim of the Workshop is to bring together academics and practitioners to discuss the complexities of testing DBMS systems and applications:

There is significant interest in testing database systems and applications within both the database and the software engineering communities. The goal of DBTest 2012 is to bring together researchers and practitioners from academia and industry to discuss key problems and ideas related to testing database systems and applications. We expect that this collaboration will facilitate the creation of research agendas and new techniques to address testing problems for database systems and database applications. The long-term objective of such work is to reduce the cost and time required to test and tune database products so that users and vendors can spend more time and energy on actual innovations.

The workshop program was finalized over the weekend by workshop co-chairs Eric Lo of Hong Kong Polytechnic and Florian Waas of EMC/Greenplum and the keynote talk by Oege de Moor, along with the 12 accepted papers, chosen from the 26 submissions, are all interesting and should make for a great workshop.

Survey to Mold "Appeon Mobile"

John Strano, PowerBuilder Evangelist — Thu, 29 Mar 2012 11:25:41 EST

Appeon Mobile Survey

Appeon Mobile will create native mobile apps using your PowerBuilder skills!

Help create a mobile solution for PowerBuilder adapted to your needs by participating in this survey!

http://tinyurl.com/AppeonSurvey

Introduction to Customization Studio 2012

John Strano, PowerBuilder Evangelist — Fri, 16 Mar 2012 00:01:26 EST

Thursday March 22nd 2012

at 12:00 PM EST (New York) - 17h00 CET (Paris)

Imagine a PowerBuilder extension that would allow end-users to modify applications themselves! Customization Studio lets them edit a window as if it were a simple MS Office document. Users modify the executable version of the application, without changing the source code, creating an immediate result.

Customization Studio 2012 features a new application called "NCS Designer"; it provides tools designed for non-technical users. In this presentation, we will see how to use these tools to customize reports, add new items in your windows, modernize the look & feel of the application and more...

Presenter: Christophe Dufourmantelle

http://powerbuilder.tv/index.php/en/upcoming-webinars/1-upcoming-webinars/152-introduction-to-customization-studio-2012

Limitations of proxy tables

Glenn Paulley, Director, Engineering — Wed, 14 Mar 2012 02:22:18 EST

Proxy tables, sometimes referred to Remote Data Access or OMNI, are a convenient way to query or modify tables in different databases all from the same connection. SQL Anywhere's proxy tables are an implementation of a loosely-coupled multidatabase system. The underlying databases do not have to be SQL Anywhere databases - any data source that supports ODBC will do, so the underlying base table for the proxy can be an Oracle table, a Microsoft SQL Server table, even an Excel spreadsheet. Once the proxy table's schema is defined in the database's catalog, the table can be queried just like any other table as if it was defined as a local table in that database.

That's the overall idea, anyway; but there are some caveats that get introduced as part of the implementation, and I'd like to speak to one of these in particular. My post is prompted by a question from a longstanding SQL Anywhere customer, Frank Vestjens, who in early February in the NNTP newsgroup sybase.public.sqlanywhere.general queried about the following SQL batch:

begin
  declare dd date;
  declare tt time;
  declare resultaat numeric;
  //
  set dd = '2012-06-07';
  set tt = '15:45:00.000';
  //
  message dd + tt type info to console;
  //
  select first Id into resultaat
  from p_mmptankplanning
  where arrivalDate + IsNull(arrivaltime,'00:00:00') <= dd+tt
  order by arrivaldate+arrivalTime,departuredate+departureTime;
end

The batch works fine with a local table p_mmptankplanning but gives an error if the table is a proxy table; the error is "Cannot convert 2012-06-0715:45:00.000 to a timestamp".

Operator overloading

In SQL Anywhere, multidatabase requests are decomposed into SQL statements that are shipped over an ODBC connection to the underlying data source. In many cases, the complete SQL statement can be shipped to the underlying server, something we call "full passthrough mode" as no post-processing is required on the originating server - the server ships the query to the underlying DBMS, and that database system returns the result set which is percolated back to the client. Since the originating server is a SQL Anywhere server, the SQL dialect of the original statement must be understood by SQL Anywhere. If the underlying DBMS isn't SQL Anywhere, then the server's Remote Data Access support may make some minor syntactic changes to the statement, or try to compensate for missing functionality in the underlying server.

The SQL statement sent to the underlying DBMS, whether or not the statement can be processed in full passthrough mode or in partial passthrough mode, is a string. Moreover, SQL Anywhere can ship SELECT, INSERT, UPDATE, DELETE and MERGE statements to the underlying DBMS - among others - but lacks the ability to ship batches or procedure definitions.

So in the query above, the problem is that the query refers to the date/time variables dd and tt, and uses the operator + to combine them into a TIMESTAMP. Since SQL Anywhere lacks the ability to ship an SQL batch, what gets shipped to the underlying DBMS server is the SQL statement

select first Id into resultaat

from p_mmptankplanning

where arrivalDate + IsNull(arrivaltime,'00:00:00') <= '2012-06-07' + '15:45:00.000'

order by arrivaldate+arrivalTime,departuredate+departureTime;

and now the problem is more evident: in SQL Anywhere, the '+' operator is overloaded to support both operations on date/time types, and on strings; with strings, '+' is string concatentation. When the statement above gets sent to the underlying SQL Anywhere server, it concatenates the two date/time strings to form the string '2012-06-0715:45:00.000' - note no intervening blank - and this leads directly to the conversion error. Robust support for SQL batches would solve the problem, but we have no plans to introduce such support at this time. A workaround is to compose the desired TIMESTAMP outside the query, so that when converted to a string the underlying query will give the desired semantics. However, even in that case care must be taken to make sure that the DATE_ORDER and DATEFORMAT option settings are compatibile across the servers involved.

My thanks to my colleague Karim Khamis for his explanations of Remote Data Access internals.

NHibernate 3.3.0CR1 released

Glenn Paulley, Director, Engineering — Tue, 13 Mar 2012 09:33:22 EST

Yesterday the first release candidate of NHibernate 3.3.0 was made available; you can find it here on sourceforge.net.

In addition to the numerous bug fixes for NHibernate's LINQ support, of interest to SQL Anywhere users will be some bug fixes and changes for SQL Anywhere 12.0.1 that I have described previously in this forum. These changes include:

A fix to the type mappings in SybaseSQLAnywhere10Dialect for binary data to LONG BINARY;
A new driver module, NHibernate.Driver.SybaseSQLAnywhereDotNet4Driver, linked to the SybaseSQLAnywhere12Dialect, to handle the renaming of the assemblies for the SQL Anywhere ADO.NET provider in SQL Anywhere version 12. Unfortunately, specifying a different assembly name for a .NET provider isn't a configuration parameter in NHibernate; the driver name is embedded directly in the NHibernate library.

Thanks to Julian Maughan for committing the changes to the NHibernate distribution.

Charlotte User Group Meeting

John Strano, PowerBuilder Evangelist — Mon, 05 Mar 2012 00:02:06 EST

March 8th, 2012 6-8pm

Open Mic' Night!

So far, topics include:

ASE/SQL Server database tips

Powerbuilder 12.5 Classic tips

PBTools application.
More to come.

The meeting will also be available as a Microsoft Livemeeting session for any virtual attendees. Your registration confirmation email will contain the email link.

Location Details:

Offices of McKesson

10735 David Taylor Drive, Suite 100

Charlotte, NC 28262

Cómo hacer aplicaciones multitarea con PowerBuilder

John Strano, PowerBuilder Evangelist — Fri, 02 Mar 2012 13:05:48 EST

Thursday March 8th 2012

at 09:00am CST (Ciudad de Mexico) / 16:00 CET (Madrid)

Cómo hacer aplicaciones multitarea con PowerBuilder

- Presented in Spanish -

En este webinar realizaremos aplicaciones multitarea en versiones de Powerbuilder anteriores a la 12, para que se puedan ejecutar como procesos independientes para aprovechar al máximo el procesador y la memoria. También explicarán las nuevas funcionalidades y ventajas del trabajo multiproceso en PowerBuilder 12.5 utilizando funciones de PowerBuilder.NET.

Presenter: Oscar Tobar

http://powerbuilder.tv/index.php/en/upcoming-webinars/1-upcoming-webinars/153-como-hacer-aplicaciones-multitarea-con-powerbuilder

Robustness and error handling

Glenn Paulley, Director, Engineering — Wed, 29 Feb 2012 14:38:29 EST

I don't usually "re-tweet" someone else's blog post. There are enough bits of ~~information~~ data flying through the Internet that I don't need to duplicate any more of them. At least, no more than absolutely necessary.

However, today I am going to make an exception and draw your attention to a recent post by Amazon Web Services VP James Hamilton entitled "Observations on Errors, Corrections, & Trust of Dependent Systems". I have known James for the better part of twenty years and his writing on infrastructure efficiency, reliability, and scaling makes for compelling reading. If you're not reading James' blog, Perspectives, on a regular basis, you should be. In part, here's what James had to say about the need for ECC memory in both client and server systems:

The immediate lesson is you absolutely do need ECC in server application[sic] and it is just about crazy to even contemplate running valuable applications without it. The extension of that learning is to ask what is really different about clients? Servers mostly have ECC but most clients don’t. On a client, each of these corrections would instead be a corruption. Client DRAM is not better and, in fact, often is worse on some dimensions. These data corruptions are happening out there on client systems every day. Each day client data is silently corrupted. Each day applications crash without obvious explanation. At scale, the additional cost of ECC asymptotically approaches the cost of the additional memory to store the ECC. I’ve argued for years that Microsoft should require ECC for Windows Hardware Certification on all systems including clients. It would be good for the ecosystem and remove a substantial source of customer frustration. In fact, it’s that observation that leads most embedded systems parts to support ECC. Nobody wants their car, camera, or TV crashing. Given the cost at scale is low, ECC memory should be part of all client systems.

James' post is timely because this week I was asked by SQL Anywhere Product Manager Eric Farrar to respond to a request from an OEM hardware infrastructure manufacturer for feedback regarding future product designs.

Other than the obvious "cheaper and faster" my wish is for one thing: robustness.

As James describes in his post, at scale, "hardware" failures are rife, causing errors, logical and physical data corruptions, system outages, crashes, you name it. I placed "hardware" in quotation marks deliberately because today's disk and flash memory hardware contains a vast quantity of software as well; the microcode for the filesystem on compact flash (CF) and SD is OEM'd and consists of thousands of lines of code. ECC correction would be nice, not only for flash or traditional magnetic media but also for RAM, as James notes. Yet even with ECC correction, things aren't that rosy, particularly with "commodity" hardware. Consider the abstract of this IEEE paper [1] from Remzi Arpaci-Dusseau's storage group at the University of Wisconsin:

We use type-aware pointer corruption to examines Windows NTFS and Linux ext3. We find that they rely on type and sanity checks to detect corruption, and NTFS recovers using replication in some instances. However, NTFS and ext3 do not recover from most corruptions, including many scenarios for which they possess sufficient redundant information, leading to further corruption, crashes, and unmountable file systems. We use our study to identify important lessons for handling corrupt pointers.

I have written about storage stack corruption at various times in the past - see here and here. That corruption need not be permanent to cause problems: logical (data) corruption caused by transient failures can be just as bad as permanent ones. James' point is that corruption detection and mitigation needs to take place in all system components, including RAM.

All of this is bad enough. Yet the situation isn't helped by the lack of standards in this area. We know from the experience of our customers that various systems fail to meet expected behaviour with respect to I/O semantics - these behaviours are, sometimes, deliberately changed in the name of better performance, but at the expense of robustness. SQL Anywhere customers are well advised to read this whitepaper entitled "SQL Anywhere I/O Requirements for Windows and Linux" for background information on what I/O semantics your server systems must support.

[1] Lakshmi N. Bairavasundaram, Meenali Rungta, Nitin Agrawal, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, and Michael M. Swift (June 2008). Analyzing the Effects of Disk Pointer Corruption. In Proceedings of the International Conference on Dependable Systems and Networks, Anchorage, Alaska, pp. 502-511.