• ERP (Enterprise Resource Planning)
• CRM (Customer Relationship Management)
• SRM (Supplier Relationship Management)
• SCM (Supply Chain Management)
• PLM (Product Lifecycle Management)

Now back to Sybase ASE: I had a look at an ASE database for SAP ERP.
Buckle up, here we go:

• SAP will be releasing Business Suite on ASE version 15.7.
• All SAP application data resides in a single ASE database. There is another small database for use by SAP tools.
• The ASE database uses a 16KB page size.
• For ERP only (i.e. not counting CRM and the other Business Suite modules), the database contains about 80,000 tables and 170,000 indexes. This is because SAP ERP has many features and functions, all with their own set of tables. SAP customers typically run only a subset of all those functions so in practice a large part of those 80,000 tables will always remain empty.
• All SAP tables use datarows locking (there is an interesting historical dimension here: check out the last big paragraph on this page)
• All tables names are in uppercase; some table names contain special characters, like the slash character in “/BCV/C_QATTR” (I don’t have a clue what that name means, BTW)
• Apart from the tables, there are also about 10,000 views. No stored procedures or triggers are used.
• SAP makes heavy use of dynamic SQL (also known as “prepared statements”).
• Many tables have a text or image column.
• All tables are owned by one database user (and that’s not the dbo user).
• The ASE database is accessed through ODBC.
• SAP makes frequent use of the built-in ASE Job Scheduler (originally added in ASE 12.5.1).
• The ASE server uses Unicode with the utf8 character set.

An architectural aspect worth mentioning is that the database is completely encapsulated in a SAP system; it runs as an embedded component.
Also, the database is always accessed through SAP tools, specifically SAP’s ‘DBA Cockpit’. DBA Cockpit is pretty much what the name suggests, namely the central management tool for an SAP system.
In principle, you would never access the database in the SAP system manually outside of DBA Cockpit. In ASE parlance: you wouldn’t use ‘isql’ in an SAP system. Things like looking at query plans or MDA tables is all done through DBA Cockpit.
In fact, I have to say that SAP’s DBA Cockpit is a really nice management and monitoring tool. Because SAP controls the application and the middleware, they have been able to build pretty nice monitoring features into it, allowing the kind of detail and drill-down that every ASE user would like to have (before you ask: yes, DBA Cockpit is specifically designed for SAP Business Suite; and no, DBA Cockpit is not available as a separate tool outside the Business Suite context).

Another question that has been asked frequently is if existing ASE DBAs will be able to help SAP customers with the implementation of Business Suite-on-ASE. My best guess is that this is will probably not be a very common sight. The primary skill needed to work with SAP customers is knowledge of the SAP application and systems architecture. Installation and creation of the ASE database is all handled by the SAP tools.
Having said that, Sybase ASE skills will definitely be needed somewhere. Customers who run SAP systems on Sybase ASE, especially for large installations, will need some DBAs with ASE knowledge.
I’ve said it before: with the SAP acquisition, having Sybase skills (ASE, IQ, Replication Server) will eventually turn out to be a Good Thing. With SAP’s recently stated ambition to become the #2 database company in a few years time (read more about that here), that’s another reason to keep your ASE skills up to date — and to migrate to ASE 15.7.

ASE 15.7 is packed with new functionality. I cannot actually remember an ASE release with so many new features as ASE 15.7. I guess that’s the reason why the webcast comes in 5 installments.
My part of the webcast was about application development features. This means: enhancements to SQL functionality or syntax whose focus is to deliver better (or more) functionality – as opposed to better performance or shorter maintenance downtime, for example (which ASE 15.7 also has, but that’s the topic of subsequent webcasts). You can listen to the webcast recording here.

Let me just briefly recap the highlights of these application-developer-oriented features in ASE 15.7.
First, there have been enhancements to handling of text/image data (collectively known as LOB data). For as long as I can remember, customers have asked for the ability to declare SQL variables and parameters of the text or image datatype. In ASE 15.7, that is now possible. This can be particularly handy when doing XML processing on mature XML documents, which can easily exceed the maximum length of 16KB for a varchar variable.
Another LOB enhancement is known as “LOB locators”. This is best described as a “pointer” to a LOB variable. When a client application wants to operate on a LOB value (like truncating/overwriting/concatenating it, etc.), ASE 15.7 can send the LOB locator to the client rather than the full LOB value, which may be big. The client app then operates on the locator with new (and existing) T-SQL statements which are sent back to the ASE server where the operation on the LOB value is effectuated. This improves efficiency since no big LOB values need to be exchanged between client and server, but only the small (24 bytes) LOB locator values instead.
If this all sounds a bit cryptic, check out the webcast slides for more detail and examples.

Another enhancement in ASE 15.7 is the MERGE statement. I have blogged about this before, so see there for more details. Suffice to say here that MERGE is a more efficient way of inserting-or-updating a bunch of rows into another table than you could ever code yourself with separate insert an update steps.
This is the point where Peter comes in. At the end of the webcast one listener with that name asked a question which I was unable to answer at the time. His question was whether you can force an index in the MERGE statement in the same way as for other types of query. After consulting with the responsible engineering team, the answer has to be that, unfortunately, this is currently not supported; the same applies to other query plan forcings (I/O size etc) as well as to specifying an abstract query plan. Should you try this, you may find that it actually works fine. However, be warned: it may also just as well not work, since there is currently a number of known issues when using these clauses for MERGE. For that reason, we recommend you do not try to use any query plan forcings around the MERGE statement for now.

There is also a new feature that is perhaps less about app development and more about performance after all. In a nutshell, by enabling the config parameter ’streamlined dynamic SQL’ (disabled by default), a series of internal optimizations around query processing becomes active (not just for dynamic SQL, but for many other types of queries too). This should translate into better performance although it is -as always- hard to predict how much gain you may see in your system. So I’m not going to speculate. Instead, please let me know what you find. There is no reason not to use this optimization, so please give it a try…

Lastly, there are some features that qualify as ’small’ in comparison to the other topics discussed. These include syntax enhancements; for example, certain constructs are now allowed in subqueries that previously caused an error.
As it happens, my favorite new ASE feature falls actually in this category. If you want to convert the number 123 to a 6-character string, and pad the result with leading ‘0′ characters (i.e. ‘000123′) in pre-15.7 you had to use the following expression (figuring out how this works is left as an exercise to you, reader):

select right(replicate('0',6) + convert(varchar,123),6)

In ASE 15.7, you can now simply specify the padding character as a third argument to the str() function, so the following is now sufficient:

select str(123, 6, '0')

Any padding character can be specified (a ‘*’ would result in ‘***123′).
Admittedly, it did probably not require a major effort by Sybase’s most talented engineers to implement this enhancement. But I always really like such little features that make my life easier when coding SQL. Small can certainly be beautiful.

(Coming up in the next webcasts: some ASE new features and enhancements that are really big. Those are cool too. Stay tuned.)

First, a quick IMDB recap: the virtue of the ASE IMDB is that there is no persistent storage: an IMDB database lives entirely in cache (=memory). If there’s no disk, then there’s no disk I/O either. Since the contents of an IMDB are lost after an ASE reboot, there is no need to do a transaction logging for the purpose of recovering the database when ASE start up again. This allows all sorts of internal optimizations around disk I/O and buffer cache management, as well as around transaction logging. Otherwise, an ASE IMDB is pretty much identical to a fully cached classic ASE database.
This all means that if you run workload that performs heavy disk write activity, and/or has many concurrent transactions, then the ASE IMDB may deliver better performance since these two areas were optimized. For other workload (like select statements), you should not expect significant performance gains from an IMDB.

Sybase’s message has always been that an ASE IMDB is functionally equivalent to any other ASE database, except that its transactions are not durable (i.e. not retained over an ASE reboot due to absence of persistent storage. This was usually the point where the question came up whether you could use ASE IMDBs with Replication Server.
As RepServer users know, replication involves a number of things in each primary or replicate database, such as: the ASE RepAgent configuration for that database, the secondary truncation point in the transaction log, the maintenance user and the rs_lastcommit table. RepServer expects these items to always be present, also when the ASE server containing the primary or replicate ASE database has been shut down and rebooted.

So far it’s all pretty obvious, but now we’ve gotten to the point where the subtleties start to kick in.

To use an IMDB database as a replication source, its secondary truncation point, rs_lastcommit table etc. need to be still present after an ASE reboot. But since we don’t have any persistent storage in an IMDB, that information will be gone after ASE restarts; consequently replication will then no longer work. To make replication to or from an IMDB possible, some non-standard setup steps are required. In addition, some internals had to be changed for use with IMDBs, both in ASE and in RepServer.
With ASE version 15.5 ESD#4 and Replication Server version 15.6 ESD#1, which were both released in July 2011, the situation is now that replicating to and from IMDBs is supported (note: you need both the ASE ESD and the RepServer ESD). Those non-standard setup steps are still required — basically this involves using a template database for the IMDB in which the truncation point, rs_lastcommit etc. are stored. This template database gets copied into the IMDB, thus providing it with the maintenance user etc. (details are described in a whitepaper; see below).

So, before ASE 15.5 ESD#4 and Replication Server 15.6 ESD#1, did replication work for IMDBs? As I mentioned earlier, the answer is both ‘yes’ and ‘no’.
The answer is ‘yes’ because when you set up the replication system, everything will keep working fine as long as ASE is not rebooted. The answer is ‘no’ because after an ASE reboot, you will need to setup the replication configuration for the IMDBs in that ASE server all over again, which is hardly how most customers would qualify as ‘replication works’.
Another ‘yes’ lies in the fact that prior to ASE 15.5 ESD#4 + RS 15.6 ESD#1, you could also use the template database to contain the truncation point etc. But, there’s another ‘no’ as well: in certain -rather specific- situations, replication from an IMDB might not always work as you would expect, and could for example cause significant delays in replicating data out of an IMDB (for internal technical reasons).

This mix of “yes, it works” and “no, it doesn’t quite work” is just a different way of saying that you really need ASE 15.5 ESD#4 as well as Replication Server 15.6 ESD#1 in order to use IMDBs in a replication system.
The details of how to set up replication for IMDBs, including actual code examples, are described in a technical whitepaper that was just published at sybase.com at this location: http://www.sybase.com/detail?id=1094415.

With that, all questions on IMDB and replication are now answered, I hope…

First, IDC analyst Carl Olofson published a report titled “SAP and Sybase: A Marriage Made in Database Heaven”. Indeed, Mr. Olofson sees some very compelling value-adds around database technology now that SAP and Sybase are the same company.
(in the interest of full disclosure, I should point out that this IDC report is an independent IDC publication, and was not sponsored by SAP or Sybase)

The IDC report points out the value of Sybase ASE as SAP’s ERP database, and therefore as a major weapon against Oracle’s dominance of the SAP database tier. This has been pointed out before, but what is interesting is to see IDC describe ASE as SAP’s ‘preferred’ database for ERP – that’s music to the ears of everyone with a stake in ASE (more on that ‘preferred’ status below).
Furthermore, Sybase IQ is seen as the preferred database to power BusinessObjects as well as non-100%-SAP enterprise data warehouses. Sybase Replication Server could be a key technology to move data into IQ as well as into HANA. That’s all good news when you have Sybase skills today.
Perhaps the most fascinating thought in the IDC report is that SAP’s in-memory HANA technology could make its way into ASE as well as IQ, thus making ASE and IQ stronger database platforms of their own. That would indeed be an enrichment for ASE and IQ.
Mr. Olofson also makes an interesting non-technical point. SAP, primarily an application-oriented company, is good at selling on business level, whereas Sybase -more an infrastructure-oriented technology company- has always been best at selling to technologists. Combining these two approaches could result in significant market power.
The IDC report is available here.

Coming back to Sybase ASE: indeed, SAP’s message is that “ASE will be SAP’s preferred database for transaction environments” (here, ‘transaction environments’ means: SAP ERP). These aren’t my words, but those of Chris McClain who is SAP’s General Manager of Mobility and Sybase Solutions. This quote comes from his SAP ERP-on-ASE presentation at the recent SAPPHIRE conference (at http://www.sapphirenow.com/sessiondetails.aspx?sId=106; jump to 02:10 should you want to hear it for yourself).
This is no small endorsement of ASE by SAP. Of course, SAP still supports all those other databases too, but now that SAP owns ASE, this brings unprecedented opportunities to optimally integrate SAP ERP with ASE. In addition, ASE is being positioned as a more cost-efficient database for ERP customers. For a business-oriented customer base like SAP’s, that is perhaps the most important selling point of all.

For me, SAP’s statements about ASE being the preferred ERP database, echoed by the IDC report, is another confirmation that the core Sybase database products enjoy fantastic opportunities as part of SAP. The future does look great indeed.

I’ve been on a road show last week (the EMEA TechDays) and on a transatlantic flight this week, and that always brings lots of wasted time at airports and on planes.
Putting some of that time to good use got me to dig into the many, many new features that will be part of ASE 15.7.
In this blog I’d just like to put the spotlight on a new feature that I really like: the MERGE SQL statement.

MERGE appeared in the ANSI SQL 2003 standard. Its can be described as “insert rows into a table, except when they already exist in that table, in which case the existing rows should be updated”.
The requirement to implement such functionality occurs more often in database systems, and is often referred to as “upsert”, i.e. “update+insert” (though, as I’ll explain later, that’s actually not a good term to use).

Here’s an example of using MERGE. We have a central table Customers that contains customer data, and we need to process a weekly set of updates. These updates are rows in the table Cust_updates_week_10 (for simplicity, the updates only concern the address and phone number in this example):

1    merge into Customers as c
2    using Cust_updates_week_10 as u
3    on c.CustID = u.CustID
4    when not matched then
5         insert (CustID,Addr,Phone) values(u.CustID,u.Addr,u.Phone)
6    when matched then
7         update set Addr=u.Addr, Phone=u.Phone

Let’s walk down the various lines:
Lines 1/2 specifies that the target is the table (or view) Customers, and the rows from Cust_updates_week_10 should be inserted or updated here.
Line 3 specifies the condition that determines whether a row exists or not. This example shows just one predicate, but you can specify more. Under the covers, ASE performs a left outer join between Cust_updates_week_10 and Customers, using this on-clause (why an outer join? Because all rows in Cust_updates_week_10 must be processed).
Line 4/5 indicate that when a row from Cust_updates_week_10 does not exist in Customers (according to the on-clause on line 3), then it is apparently a ‘new’ customer and it must be inserted.
If the row does exist already (line 6), then the non-key columns of the existing row must be updated (line 7 — again, this example restricts itself to address and phone number columns only)

The basic idea of the MERGE syntax can be extended. For example, there can be multiple “when matched” and “when not matched” clauses, and these can specify additional predicates.
In the example below, there are two “when matched” clauses, with the first one testing for an empty address — meaning that when the row’s key already exists, but the new address is empty, then the existing row must be deleted from the Customers table:

merge into Customers as c
using Cust_updates_week_10 as u
on c.CustID = u.CustID
when not matched then
  insert (CustID,Addr,Phone) values(u.CustID,u.Addr,u.Phone)
when matched and u.Addr = NULL then
  delete
when matched then
  update set Addr=u.Addr, Phone=u.Phone

Note that the order of multiple “when matched” or “when not matched” clauses is important: for the first clause where all conditions are true, the action is executed, and subsequent clauses of the same “when [not] matched” type, will be ignored.

This last example shows why “upsert” is too restricted a summary of what MERGE can do: not just insert-or-otherwise-update, but it can also delete. So “updelsert” would do more justice to the functionality of MERGE, but somehow that sounds less catch than “upsert”.

The attraction or MERGE is that you can pack quite a bit of functionality into one statement. Not only does that let you write more compact SQL code, but it can also deliver better performance. The MERGE statement makes only one pass over the source table (Cust_updates_week_10), and directly processes every row it finds. If you were to write the same functionality with individual insert, update and delete statements, you would need at least three passes over the data.

MERGE is flexible: instead of using a source table, a derived table can be specified as follows:

merge into Customers as c
using (select CustID, Addr, Phone from Cust_updates_week_10
       where Phone is not NULL) as u
on c.CustID = u.CustID
when not matched then
  insert (CustID,Addr,Phone) values(u.CustID,u.Addr,u.Phone
when matched then
  update set Addr=u.Addr, Phone=u.Phone

Note that there are some restrictions around using MERGE. For example, the key columns on line 3 cannot be updated. There may be some other restrictions too; we’ll see how far things got once 15.7 is released.

Stay tuned for more information about ASE 15.7!

Making the best of my somewhat handicapped situation got me to focus more on reading than on anything else for the past month. (I have this mail folder labelled “MUST-READ” for interesting stuff I come across that I really need to read, but somehow that folder just keeps filling up. The information age is great with all that interesting and highly relevant material you can find quickly, but digesting it still takes old-fashioned time. Why do I never hear about a solution for that problem? But I digress again.)

Anyway, after catching up on some outstanding material I realised that 2011 promises to see quite some interesting stuff happening in the Sybase database arena.

First, ASE (Adaptive Server Enterprise, Sybase’s mission-critical OLTP database) will be making its debut as the database powering SAP’s Business Suite (formerly known as R/3) later this year. This should be great news for everyone with a stake or interest in ASE, as this constitutes a major endorsement of ASE. It also validates ASE’s long-standing approach towards resource-efficiency, low TCO, and high OLTP performance. In my opinion, SAP’s embracing of ASE as the database engine for their main product can only strengthen ASE as a platform. Hopefully, at some point this translates into a richer ecosystem around ASE, more ISVs certifying their applications on ASE, and more demand for ASE skills.
Furthermore, a fresh ASE release will likely appear somewhere in 2011, featuring an interesting range of new features and functionality. Stay tuned for more details.

Another bright spot in the sky is the upcoming version 15.3 of Sybase IQ, our analytics database. As you may have heard, the beta version of IQ 15.3 was released a while ago. Loosely put, IQ 15.3 can use multiple cluster nodes to process a single query. Previously, when an IQ cluster node received a query, it had to process it entirely by itself. However, in IQ 15.3, capacity from other cluster nodes can also be used to process that query. This allows IQ to put the hardware’s computing power to maximum use, making IQ a more flexible, powerful and efficient analytics engine. I must admit I didn’t get round to trying out IQ 15.3 myself yet (an issue temporarily inhibiting my typing ability, which I may have mentioned), but I’m certainly eager.
Other good news around IQ are the announced plans to provide closer integration between Business Objects (also an SAP company) and IQ. While no specifics have yet been announced, the prospect of BOBJ specifically optimised for IQ (or the other way around) feels quite compelling: this combination would really deliver on the promise to extract value from data in less time.

Interesting smells are coming out of the SQL Anywhere kitchen, too. Some recent blog posts indicate some interesting ideas around cloud-enabled database processing. While I’m usually not much into SQL Anywhere (I’m really an ASE guy), this did attract my attention. I’ll be closely following what the SQL Anywhere folks will be serving up, and I recommend you do the same.

And finally, another recommendation: be careful with that gravity stuff. It can pull you down really hard.