UpSizeR: Synthetically Scaling Up a Given Database State

May 27, 2010
Y.C. Tay | National University of Singapore

E-commerce and social networking services must ensure that their systems are scalable. Engineering for rapid growth requires intensive testing with scaled-up datasets. Although such a larger dataset is synthetically generated, it must be similar to a real dataset if it is to be useful.

This talk presents UpSizeR, a tool for scaling up relational databases. Given a database state D and a positive number s, UpSizeR generates a synthetic state D’ that is s times the size of D, yet similar to D in terms of query results. UpSizeR does this by extracting inter-column and inter-row information from D. UpSizeR can also be used by an enterprise to make a synthetic copy (s=1) of its proprietary dataset for a vendor, or scale down a production dataset (s<1) for non-production testing. Experiments with Flickr data shows good agreement between crawled data and UpSizeR output for various sizes.

However, UpSizeR currently cannot scale the social network topology in Flickr. This leads to the Attribute Value Correlation Problem: If D records data from a social network, how do the social interactions affect correlation among attribute values in D?

Speaker Details

Y.C. Tay received his BSc degree from the University of Singapore and PhD degree from Harvard University. He is a faculty member at the National University of Singapore (http://www.math.nus.edu.sg/~mattyc), currently on sabbatical at Microsoft. His main research interest is performance modeling (transaction processing, multimedia load-sharing, parallel rendering and wireless protocols). Other recent interests are: correctness in distributed and parallel computing, routing protocols for mobility support and ad hoc wireless networks, and application of data mining to online optimization.