Incremental window aggregates over array database

Li Jiang; Hideyuki Kawashima; Osamu Tatebe

doi:10.1109/BigData.2014.7004230

Incremental window aggregates over array database

Li Jiang, Hideyuki Kawashima, Osamu Tatebe

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

We propose an efficient window aggregation method over multi-dimensional array data based on incremental computation. We improve several aggregations with different data structures exploited to achieve efficient computation: list for sum and avg, heap for max and min, and balanced binary search tree for percentile. We present time complexity analysis for the methods, and then evaluate performance with experiments in SciDB array database system with both synthetic and JRA55 meteorological dataset. Our analysis shows that performance improvement is proportional to the window size in the last dimension in theory, and the result of experiment is consistent with the analysis. In certain cases, it shows an acceleration factor more than 13 by the proposed method with percentile, while a factor over 28 with maximum.

Original language	English
Title of host publication	Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014
Editors	Wo Chang, Jun Huan, Nick Cercone, Saumyadipta Pyne, Vasant Honavar, Jimmy Lin, Xiaohua Tony Hu, Charu Aggarwal, Bamshad Mobasher, Jian Pei, Raghunath Nambiar
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	183-188
Number of pages	6
ISBN (Electronic)	9781479956654
DOIs	https://doi.org/10.1109/BigData.2014.7004230
Publication status	Published - 2015 Jan 7
Externally published	Yes
Event	2nd IEEE International Conference on Big Data, IEEE Big Data 2014 - Washington, United States Duration: 2014 Oct 27 → 2014 Oct 30

Publication series

Name	Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

Other

Other	2nd IEEE International Conference on Big Data, IEEE Big Data 2014
Country/Territory	United States
City	Washington
Period	14/10/27 → 14/10/30

Keywords

Array Database
Incremental Computation
Multi-Dimensional Array
Window Aggregates

ASJC Scopus subject areas

Artificial Intelligence
Information Systems

Access to Document

10.1109/BigData.2014.7004230

Cite this

Jiang, L., Kawashima, H., & Tatebe, O. (2015). Incremental window aggregates over array database. In W. Chang, J. Huan, N. Cercone, S. Pyne, V. Honavar, J. Lin, X. T. Hu, C. Aggarwal, B. Mobasher, J. Pei, & R. Nambiar (Eds.), Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014 (pp. 183-188). Article 7004230 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2014.7004230

Incremental window aggregates over array database. / Jiang, Li; Kawashima, Hideyuki; Tatebe, Osamu.
Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. ed. / Wo Chang; Jun Huan; Nick Cercone; Saumyadipta Pyne; Vasant Honavar; Jimmy Lin; Xiaohua Tony Hu; Charu Aggarwal; Bamshad Mobasher; Jian Pei; Raghunath Nambiar. Institute of Electrical and Electronics Engineers Inc., 2015. p. 183-188 7004230 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jiang, L, Kawashima, H & Tatebe, O 2015, Incremental window aggregates over array database. in W Chang, J Huan, N Cercone, S Pyne, V Honavar, J Lin, XT Hu, C Aggarwal, B Mobasher, J Pei & R Nambiar (eds), Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014., 7004230, Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014, Institute of Electrical and Electronics Engineers Inc., pp. 183-188, 2nd IEEE International Conference on Big Data, IEEE Big Data 2014, Washington, United States, 14/10/27. https://doi.org/10.1109/BigData.2014.7004230

Jiang L, Kawashima H, Tatebe O. Incremental window aggregates over array database. In Chang W, Huan J, Cercone N, Pyne S, Honavar V, Lin J, Hu XT, Aggarwal C, Mobasher B, Pei J, Nambiar R, editors, Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. Institute of Electrical and Electronics Engineers Inc. 2015. p. 183-188. 7004230. (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014). doi: 10.1109/BigData.2014.7004230

Jiang, Li ; Kawashima, Hideyuki ; Tatebe, Osamu. / Incremental window aggregates over array database. Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014. editor / Wo Chang ; Jun Huan ; Nick Cercone ; Saumyadipta Pyne ; Vasant Honavar ; Jimmy Lin ; Xiaohua Tony Hu ; Charu Aggarwal ; Bamshad Mobasher ; Jian Pei ; Raghunath Nambiar. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 183-188 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014).

@inproceedings{8d7c8e16a2ac4cf288607ad9487772ea,

title = "Incremental window aggregates over array database",

abstract = "We propose an efficient window aggregation method over multi-dimensional array data based on incremental computation. We improve several aggregations with different data structures exploited to achieve efficient computation: list for sum and avg, heap for max and min, and balanced binary search tree for percentile. We present time complexity analysis for the methods, and then evaluate performance with experiments in SciDB array database system with both synthetic and JRA55 meteorological dataset. Our analysis shows that performance improvement is proportional to the window size in the last dimension in theory, and the result of experiment is consistent with the analysis. In certain cases, it shows an acceleration factor more than 13 by the proposed method with percentile, while a factor over 28 with maximum.",

keywords = "Array Database, Incremental Computation, Multi-Dimensional Array, Window Aggregates",

author = "Li Jiang and Hideyuki Kawashima and Osamu Tatebe",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 2nd IEEE International Conference on Big Data, IEEE Big Data 2014 ; Conference date: 27-10-2014 Through 30-10-2014",

year = "2015",

month = jan,

day = "7",

doi = "10.1109/BigData.2014.7004230",

language = "English",

series = "Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "183--188",

editor = "Wo Chang and Jun Huan and Nick Cercone and Saumyadipta Pyne and Vasant Honavar and Jimmy Lin and Hu, {Xiaohua Tony} and Charu Aggarwal and Bamshad Mobasher and Jian Pei and Raghunath Nambiar",

booktitle = "Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014",

}

TY - GEN

T1 - Incremental window aggregates over array database

AU - Jiang, Li

AU - Kawashima, Hideyuki

AU - Tatebe, Osamu

PY - 2015/1/7

Y1 - 2015/1/7

N2 - We propose an efficient window aggregation method over multi-dimensional array data based on incremental computation. We improve several aggregations with different data structures exploited to achieve efficient computation: list for sum and avg, heap for max and min, and balanced binary search tree for percentile. We present time complexity analysis for the methods, and then evaluate performance with experiments in SciDB array database system with both synthetic and JRA55 meteorological dataset. Our analysis shows that performance improvement is proportional to the window size in the last dimension in theory, and the result of experiment is consistent with the analysis. In certain cases, it shows an acceleration factor more than 13 by the proposed method with percentile, while a factor over 28 with maximum.

AB - We propose an efficient window aggregation method over multi-dimensional array data based on incremental computation. We improve several aggregations with different data structures exploited to achieve efficient computation: list for sum and avg, heap for max and min, and balanced binary search tree for percentile. We present time complexity analysis for the methods, and then evaluate performance with experiments in SciDB array database system with both synthetic and JRA55 meteorological dataset. Our analysis shows that performance improvement is proportional to the window size in the last dimension in theory, and the result of experiment is consistent with the analysis. In certain cases, it shows an acceleration factor more than 13 by the proposed method with percentile, while a factor over 28 with maximum.

KW - Array Database

KW - Incremental Computation

KW - Multi-Dimensional Array

KW - Window Aggregates

UR - http://www.scopus.com/inward/record.url?scp=84921756594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84921756594&partnerID=8YFLogxK

U2 - 10.1109/BigData.2014.7004230

DO - 10.1109/BigData.2014.7004230

M3 - Conference contribution

AN - SCOPUS:84921756594

T3 - Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

SP - 183

EP - 188

BT - Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

A2 - Chang, Wo

A2 - Huan, Jun

A2 - Cercone, Nick

A2 - Pyne, Saumyadipta

A2 - Honavar, Vasant

A2 - Lin, Jimmy

A2 - Hu, Xiaohua Tony

A2 - Aggarwal, Charu

A2 - Mobasher, Bamshad

A2 - Pei, Jian

A2 - Nambiar, Raghunath

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2nd IEEE International Conference on Big Data, IEEE Big Data 2014

Y2 - 27 October 2014 through 30 October 2014

ER -

Incremental window aggregates over array database

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this