Efficient Multi-way Theta-Join Processing Using MapReduce

Abstract: Multi-way Theta-join queries are powerful in describing complex relations and therefore widely employed in real practices. However, existing solutions from traditional distributed and parallel databases for multi-way Theta-join queries cannot be easily extended to fit a shared-nothing distributed computing paradigm, which is proven to be able to support OLAP applications over immense data volumes. In this work, we study the problem of efficient processing of multi-way Theta-join queries using MapReduce from a cost-effective perspective. Although there have been some works using the (key,value) pair-based programming model to support join operations, efficient processing of multi-way Theta-join queries has never been fully explored. The substantial challenge lies in, given a number of processing units (that can run Map or Reduce tasks), mapping a multi-way Theta-join query to a number of MapReduce jobs and having them executed in a well scheduled sequence, such that the total processing time span is minimized. Our solution mainly includes two parts: 1) cost metrics for both single MapReduce job and a number of MapReduce jobs executed in a certain order; 2) the efficient execution of a chain-typed Theta-join with only one MapReduce job. Comparing with the query evaluation strategy proposed in [23] and the widely adopted Pig Latin and Hive SQL solutions, our method achieves significant improvement of the join processing efficiency.


Stubby: A Transformation-based Optimizer for MapReduce Workflows

Abstract: There is a growing trend of performing analysis on large datasets using workflows composed of MapReduce jobs connected through producer-consumer relationships based on data. This trend has spurred the development of a number of interfaces--ranging from program-based to query-based interfaces--for generating MapReduce workflows. Studies have shown that the gap in performance can be quite large between optimized and unoptimized workflows. However, automatic cost-based optimization of MapReduce workflows remains a challenge due to the multitude of interfaces, large size of the execution plan space, and the frequent unavailability of all types of information needed for optimization. We introduce a comprehensive plan space for MapReduce workflows generated by popular workflow generators. We then propose Stubby, a cost-based optimizer that searches selectively through the subspace of the full plan space that can be enumerated correctly and costed based on the information available in any given setting. Stubby enumerates the plan space based on plan-to-plan transformations and an efficient search algorithm. Stubby is designed to be extensible to new interfaces and new types of optimizations, which is a desirable feature given how rapidly MapReduce systems are evolving. Stubby's efficiency and effectiveness have been evaluated using representative workflows from many domains.


Labeling Workflow Views with Fine-Grained Dependencies

Abstract: This paper considers the problem of efficiently answering reachability queries over views of provenance graphs, derived from executions of workflows that may include recursion. Such views include composite modules and model fine-grained dependencies between module inputs and outputs. A novel view-adaptive dynamic labeling scheme is developed for efficient query evaluation, in which view specifications are labeled statically (i.e. as they are created) and data items are labeled dynamically as they are produced during a workflow execution. Although the combination of fine-grained dependencies and recursive workflows entail, in general, long (linear-size) data labels, we show that for a large natural class of workflows and views, labels are compact (logarithmic-size) and reachability queries can be evaluated in constant time. Experimental results demonstrate the benefit of this approach over the state-of-the-art technique when applied for labeling multiple views.


Fundamentals of Order Dependencies

Abstract: Dependencies have played a significant role in database design for many years. They have also been shown to be useful in query optimization. In this paper, we discuss dependencies between lexicographically ordered sets of tuples. We introduce formally the concept of order dependency and present a set of axioms (inference rules) for them. We show how query rewrites based on these axioms can be used for query optimization. We present several interesting theorems that can be derived using the inference rules. We prove that functional dependencies are subsumed by order dependencies and that our set of axioms for order dependencies is sound and complete.


Optimization of Analytic Window Functions

Abstract: Analytic functions represent the state-of-the-art way of performing complex data analysis within a single SQL statement. In particular, an important class of analytic functions that has been frequently used in commercial systems to support OLAP and decision support applications is the class of window functions. A window function returns for each input tuple a value derived from applying a function over a window of neighboring tuples. However, existing window function evaluation approaches are based on a naive sorting scheme. In this paper, we study the problem of optimizing the evaluation of window functions. We propose several efficient techniques, and identify optimization opportunities that allow us to optimize the evaluation of a set of window functions. We have integrated our scheme into PostgreSQL. Our comprehensive experimental study on the TPC-DS datasets as well as synthetic datasets and queries demonstrate significant speedup over existing approaches.


Opening the Black Boxes in Data Flow Optimization

Abstract: Many systems for big data analytics employ a data flow abstraction to define parallel data processing tasks. In this setting, custom operations expressed as user-defined functions are very common. We address the problem of performing data flow optimization at this level of abstraction, where the semantics of operators are not known. Traditionally, query optimization is applied to queries with known algebraic semantics. In this work, we find that a handful of properties, rather than a full algebraic specification, suffice to establish reordering conditions for data processing operators. We show that these properties can be accurately estimated for black box operators by statically analyzing the general-purpose code of their user-defined functions. We design and implement an optimizer for parallel data flows that does not assume knowledge of semantics or algebraic properties of operators. Our evaluation confirms that the optimizer can apply common rewritings such as selection reordering, bushy join-order enumeration, and limited forms of aggregation push-down, hence yielding similar rewriting power as modern relational DBMS optimizers. Moreover, it can optimize the operator order of non-relational data flows, a unique feature among today's systems.


Spinning Fast Iterative Data Flows

Abstract: Parallel dataflow systems are a central part of most analytic pipelines for big data. The iterative nature of many analysis and machine learning algorithms, however, is still a challenge for current systems. While certain types of bulk iterative algorithms are supported by novel dataflow frameworks, these systems cannot exploit computational dependencies present in many algorithms, such as graph algorithms. As a result, these algorithms are inefficiently executed and have led to specialized systems based on other paradigms, such as message passing or shared memory. We propose a method to integrate incremental iterations, a form of workset iterations, with parallel dataflows. After showing how to integrate bulk iterations into a dataflow system and its optimizer, we present an extension to the programming model for incremental iterations. The extension alleviates for the lack of mutable state in dataflows and allows for exploiting the sparse computational dependencies inherent in many iterative algorithms. The evaluation of a prototypical implementation shows that those aspects lead to up to two orders of magnitude speedup in algorithm runtime, when exploited. In our experiments, the improved dataflow system is highly competitive with specialized systems while maintaining a transparent and unified dataflow abstraction.


REX: Recursive, Delta-Based Data-Centric Computation

Abstract: In today's Web and social network environments, query workloads include ad hoc and OLAP queries, as well as iterative algorithms that analyze data relationships (e.g., link analysis, clustering, learning). Modern DBMSs support ad hoc and OLAP queries, but most are not robust enough to scale to large clusters. Conversely, "cloud" platforms like MapReduce execute chains of batch tasks across clusters in a fault tolerant way, but have too much overhead to support ad hoc queries. Moreover, both classes of platform incur significant overhead in executing iterative data analysis algorithms. Most such iterative algorithms repeatedly refine portions of their answers, until some convergence criterion is reached. However, general cloud platforms typically must reprocess all data in each step. DBMSs that support recursive SQL are more efficient in that they propagate only the changes in each step -- but they still accumulate each iteration's state, even if it is no longer useful. User-defined functions are also typically harder to write for DBMSs than for cloud platforms. We seek to unify the strengths of both styles of platforms, with a focus on supporting iterative computations in which changes, in the form of deltas, are propagated from iteration to iteration, and state is efficiently updated in an extensible way. We present a programming model oriented around deltas, describe how we execute and optimize such programs in our REX runtime system, and validate that our platform also handles failures gracefully. We experimentally validate our techniques, and show speedups over the competing methods ranging from 2.5 to nearly 100 times.


K-Reach: Who is in Your Small World

Abstract: We study the problem of answering k-hop reachability queries in a directed graph, i.e., whether there exists a directed path of length k, from a source query vertex to a target query vertex in the input graph. The problem of k-hop reachability is a general problem of the classic reachability (where k=infinity). Existing indexes for processing classic reachability queries, as well as for processing shortest path queries, are not applicable or not efficient for processing k-hop reachability queries. We propose an index for processing k-hop reachability queries, which is simple in design and efficient to construct. Our experimental results on a wide range of real datasets show that our index is more efficient than the state-of-the-art indexes even for processing classic reachability queries, for which these indexes are primarily designed. We also show that our index is efficient in answering k-hop reachability queries.


Performance Guarantees for Distributed Reachability Queries

Abstract: In the real world a graph is often fragmented and distributed across different sites. This highlights the need for evaluating queries on distributed graphs. This paper proposes distributed evaluation algorithms for three classes of queries: reachability for determining whether one node can reach another, bounded reachability for deciding whether there exists a path of a bounded length between a pair of nodes, and regular reachability for checking whether there exists a path connecting two nodes such that the node labels on the path form a string in a given regular expression. We develop these algorithms based on partial evaluation, to explore parallel computation. When evaluating a query Q on a distributed graph G, we show that these algorithms possess the following performance guarantees, no matter how G is fragmented and distributed: (1) each site is visited only once; (2) the total network traffic is determined by the size of Q and the fragmentation of G, independent of the size of G; and (3) the response time is decided by the largest fragment of G rather than the entire G. In addition, we show that these algorithms can be readily implemented in the MapReduce framework. Using synthetic and real-life data, we experimentally verify that these algorithms are scalable on large graphs, regardless of how the graphs are distributed.


Efficient Indexing and Querying over Syntactically Annotated Trees

Abstract: Natural language text corpora are often available as sets of syntactically parsed trees. A wide range of expressive tree queries are possible over such parsed trees that open a new avenue in searching over natural language text. They not only allow for querying roles and relationships within sentences, but also improve search effectiveness compared to flat keyword queries. One major drawback of current systems supporting querying over parsed text is the performance of evaluating queries over large data. In this paper we propose a novel indexing scheme over unique subtrees as index keys. We also propose a novel root-split coding scheme that stores subtree structural information only partially, thus reducing index size and improving querying performance. Our extensive set of experiments show that root-split coding reduces the index size of any interval coding which stores individual node numbers by a factor of 50% to 80%, depending on the sizes of subtrees indexed. Moreover, We show that our index using root-split coding, outperforms previous approaches by at least an order of magnitude in terms of the response time of queries.


PrivBasis: Frequent Itemset Mining with Differential Privacy

Abstract: The discovery of frequent itemsets can serve valuable economic and research purposes. Releasing discovered frequent itemsets, however, presents privacy challenges. In this paper, we study the problem of how to perform frequent itemset mining on transaction databases while satisfying differential privacy. We propose an approach, called PrivBasis, which leverages a novel notion called basis sets. A theta-basis set has the property that any itemset with frequency higher than theta is a subset of some basis. We introduce algorithms for privately constructing a basis set and then using it to find the most frequent itemsets. Experiments show that our approach greatly outperforms the current state of the art.


Low-Rank Mechanism: Optimizing Batch Queries under Differential Privacy

Abstract: Differential privacy is a promising privacy-preserving paradigm for statistical query processing over sensitive data. It works by injecting random noise into each query result, such that it is provably hard for the adversary to infer the presence or absence of any individual record from the published noisy results. The main objective in differentially private query processing is to maximize the accuracy of the query results, while satisfying the privacy guarantees. Previous work, notably the matrix mechanism, has suggested that processing a batch of correlated queries as a whole can potentially achieve considerable accuracy gains, compared to answering them individually. However, as we point out in this paper, the matrix mechanism is mainly of theoretical interest; in particular, several inherent problems in its design limit its accuracy in practice, which almost never exceeds that of naive methods. In fact, we are not aware of any existing solution that can effectively optimize a query batch under differential privacy. Motivated by this, we propose the Low-Rank Mechanism (LRM), the first practical differentially private technique for answering batch queries with high accuracy, based on a low rank approximation of the workload matrix. We prove that the accuracy provided by LRM is close to the theoretical lower bound for any mechanism to answer a batch of queries under differential privacy. Extensive experiments using real data demonstrate that LRM consistently outperforms state-of-the-art query processing solutions under differential privacy, by large margins.


Search Me If You Can: Privacy-preserving Location Query Service

Abstract: Location-Based Service (LBS) becomes increasingly popular with the dramatic growth of smartphones and social network services (SNS), and its context-rich functionalities attract considerable users. Many LBS providers use users' location information to offer them convenience and useful functions. However, the LBS could greatly breach personal privacy because location itself contains much information. Hence, preserving location privacy while achieving utility from it is still an challenging question now. This paper tackles this non-trivial challenge by designing a suite of novel fine-grained Privacy-preserving Location Query Protocol (PLQP). Our protocol allows different levels of location query on encrypted location information for different users, and it is efficient enough to be applied in mobile platforms.


Analysis of access in the Take-Grant model

Abstract: The article discribe methods of verifing the conditions of access in computer systems based on Take-Grant protection model.


Truthful Auction Mechanism for Heterogeneous Spectrum Allocation in Wireless Networks

Abstract: Secondary spectrum auction is widely applied in wireless networks for mitigating the spectrum scarcity. In a realistic spectrum trading market, the requests from secondary users often specify the usage of a fixed spectrum frequency band in a certain geographical region and require a duration time in a fixed available time interval. Considering the selfish behaviors of secondary users, it is imperative to design a truthful auction which matches the available spectrums and requests of secondary users optimally. Unfortunately, existing designs either do not consider spectrum heterogeneity or ignore the differences of required time among secondary users. In this paper, we address this problem by investigating how to use auction mechanisms to allocate and price spectrum resources so that the social efficiency can be maximized. We begin by classifying the spectrums and requests from secondary users into different local markets which ensures there is no interference between local markets, and then we can focus on the auction in a single local market. We first design an optimal auction based on the Vickrey-Clarke-Groves (VCG) mechanism to maximize the social efficiency while enforcing truthfulness. To reduce the computational complexity, we further propose a truthful sub-optimal auction with polynomial time complexity, which yields an approximation factor 6+4\surd2. Our extensive simulation results using real spectrum availability data show that the social efficiency ratio of the sub-optimal auction is always above 70% compared with the optimal auction.


Personalization in Geographic information systems: A survey

Abstract: Geographic Information Systems (GIS) are widely used in different domains of applications, such as maritime navigation, museums visits and route planning, as well as ecological, demographical and economical applications. Nowadays, organizations need sophisticated and adapted GIS-based Decision Support System (DSS) to get quick access to relevant information and to analyze data with respect to geographic information, represented not only as spatial objects, but also as maps. Several research works on GIS personalization was proposed: Face the great challenge of developing both the theory and practice to provide personalization GIS visualization systems. This paper aims to provide a comprehensive review of literature on presented GIS personalization approaches. A benchmarking study of GIS personalization methods is proposed. Several evaluation criteria are used to identify the existence of trends as well as potential needs for further investigations.


Spatial and Spatio-Temporal Multidimensional Data Modelling: A Survey

Abstract: Data warehouse store and provide access to large volume of historical data supporting the strategic decisions of organisations. Data warehouse is based on a multidimensional model which allow to express user's needs for supporting the decision making process. Since it is estimated that 80% of data used for decision making has a spatial or location component [1, 2], spatial data have been widely integrated in Data Warehouses and in OLAP systems. Extending a multidimensional data model by the inclusion of spatial data provides a concise and organised spatial datawarehouse representation. This paper aims to provide a comprehensive review of litterature on developed and suggested spatial and spatio-temporel multidimensional models. A benchmarking study of the proposed models is presented. Several evaluation criterias are used to identify the existence of trends as well as potential needs for further investigations.


Naming and Counting in Anonymous Unknown Dynamic Networks

Abstract: In this work, we study the fundamental naming and counting problems (and some variations) in networks that are anonymous, unknown, and possibly dynamic. In counting, nodes must determine the size of the network n and in naming they must end up with unique identities. By anonymous we mean that all nodes begin from identical states apart possibly from a unique leader node and by unknown that nodes have no a priori knowledge of the network (apart from some minimal knowledge when necessary) including ignorance of n. Network dynamicity is modeled by the 1-interval connectivity model, in which communication is synchronous and a worst-case adversary chooses the edges of every round subject to the condition that each instance is connected. We first focus on static networks with broadcast where we prove that, without a leader, counting is impossible to solve and that naming is impossible to solve even with a leader and even if nodes know n. These impossibilities carry over to dynamic networks as well. We also show that a unique leader suffices in order to solve counting in linear time. Then we focus on dynamic networks with broadcast. We conjecture that dynamicity renders nontrivial computation impossible. In view of this, we let the nodes know an upper bound on the maximum degree that will ever appear and show that in this case the nodes can obtain an upper bound on n. Finally, we replace broadcast with one-to-each, in which a node may send a different message to each of its neighbors. Interestingly, this natural variation is proved to be computationally equivalent to a full-knowledge model, in which unique names exist and the size of the network is known.


Opportunistic Forwarding with Partial Centrality

Abstract: In opportunistic networks, the use of social metrics (e.g., degree, closeness and betweenness centrality) of human mobility network, has recently been shown to be an effective solution to improve the performance of opportunistic forwarding algorithms. Most of the current social-based forwarding schemes exploit some globally defined node centrality, resulting in a bias towards the most popular nodes. However, these nodes may not be appropriate relay candidates for some target nodes, because they may have low importance relative to these subsets of target nodes. In this paper, to improve the opportunistic forwarding efficiency, we exploit the relative importance (called partial centrality) of a node with respect to a group of nodes. We design a new opportunistic forwarding scheme, opportunistic forwarding with partial centrality (OFPC), and theoretically quantify the influence of the partial centrality on the data forwarding performance using graph spectrum. By applying our scheme on three real opportunistic networking scenarios, our extensive evaluations show that our scheme achieves significantly better mean delivery delay and cost compared to the state-of-the-art works, while achieving delivery ratios sufficiently close to those by Epidemic under different TTL requirements.


Adaptation of pedagogical resources description standard (LOM) with the specificity of Arabic language

Abstract: In this article we focus firstly on the principle of pedagogical indexing and characteristics of Arabic language and secondly on the possibility of adapting the standard for describing learning resources used (the LOM and its Application Profiles) with learning conditions such as the educational levels of students and their levels of understanding,... the educational context with taking into account the representative elements of text, text length, ... in particular, we put in relief the specificity of the Arabic language which is a complex language, characterized by its flexion, its voyellation and agglutination.


The Complexity of MaxMin Length Triangulation

Abstract: In 1991, Edelsbrunner and Tan gave an O(n^2) algorithm for finding the MinMax Length triangulation of a set of points in the plane. In this paper we resolve one of the open problems stated in that paper, by showing that finding a MaxMin Length triangulation is an NP-complete problem. The proof implies that (unless P=NP), there is no polynomial-time approximation algorithm that can approximate the problem within any polynomial factor.


Towards the Next Generation of Data Warehouse Personalization System: A Survey and a Comparative Study

Abstract: Multidimensional databases are a great asset for decision making. Their users express complex OLAP (On-Line Analytical Processing) queries, often returning huge volumes of facts, sometimes providing little or no information. Furthermore, due to the huge volume of historical data stored in DWs, the OLAP applications may return a big amount of irrelevant information that could make the data exploration process not efficient and tardy. OLAP personalization systems play a major role in reducing the effort of decision-makers to find the most interesting information. Several works dealing with OLAP personalization were presented in the last few years. This paper aims to provide a comprehensive review of literature on OLAP personalization approaches. A benchmarking study of OLAP personalization methods is proposed. Several evaluation criteria are used to identify the existence of trends as well as potential needs for further investigations.


Functional Mechanism: Regression Analysis under Differential Privacy

Abstract: \epsilon-differential privacy is the state-of-the-art model for releasing sensitive information while protecting privacy. Numerous methods have been proposed to enforce epsilon-differential privacy in various analytical tasks, e.g., regression analysis. Existing solutions for regression analysis, however, are either limited to non-standard types of regression or unable to produce accurate regression results. Motivated by this, we propose the Functional Mechanism, a differentially private method designed for a large class of optimization-based analyses. The main idea is to enforce epsilon-differential privacy by perturbing the objective function of the optimization problem, rather than its results. As case studies, we apply the functional mechanism to address two most widely used regression models, namely, linear regression and logistic regression. Both theoretical analysis and thorough experimental evaluations show that the functional mechanism is highly effective and efficient, and it significantly outperforms existing solutions.


Publishing Microdata with a Robust Privacy Guarantee

Abstract: Today, the publication of microdata poses a privacy threat. Vast research has striven to define the privacy condition that microdata should satisfy before it is released, and devise algorithms to anonymize the data so as to achieve this condition. Yet, no method proposed to date explicitly bounds the percentage of information an adversary gains after seeing the published data for each sensitive value therein. This paper introduces beta-likeness, an appropriately robust privacy model for microdata anonymization, along with two anonymization schemes designed therefor, the one based on generalization, and the other based on perturbation. Our model postulates that an adversary's confidence on the likelihood of a certain sensitive-attribute (SA) value should not increase, in relative difference terms, by more than a predefined threshold. Our techniques aim to satisfy a given beta threshold with little information loss. We experimentally demonstrate that (i) our model provides an effective privacy guarantee in a way that predecessor models cannot, (ii) our generalization scheme is more effective and efficient in its task than methods adapting algorithms for the k-anonymity model, and (iii) our perturbation method outperforms a baseline approach. Moreover, we discuss in detail the resistance of our model and methods to attacks proposed in previous research.


Measuring Two-Event Structural Correlations on Graphs

Abstract: Real-life graphs usually have various kinds of events happening on them, e.g., product purchases in online social networks and intrusion alerts in computer networks. The occurrences of events on the same graph could be correlated, exhibiting either attraction or repulsion. Such structural correlations can reveal important relationships between different events. Unfortunately, correlation relationships on graph structures are not well studied and cannot be captured by traditional measures. In this work, we design a novel measure for assessing two-event structural correlations on graphs. Given the occurrences of two events, we choose uniformly a sample of "reference nodes" from the vicinity of all event nodes and employ the Kendall's tau rank correlation measure to compute the average concordance of event density changes. Significance can be efficiently assessed by tau's nice property of being asymptotically normal under the null hypothesis. In order to compute the measure in large scale networks, we develop a scalable framework using different sampling strategies. The complexity of these strategies is analyzed. Experiments on real graph datasets with both synthetic and real events demonstrate that the proposed framework is not only efficacious, but also efficient and scalable.


Ranking Large Temporal Data

Abstract: Ranking temporal data has not been studied until recently, even though ranking is an important operator (being promoted as a firstclass citizen) in database systems. However, only the instant top-k queries on temporal data were studied in, where objects with the k highest scores at a query time instance t are to be retrieved. The instant top-k definition clearly comes with limitations (sensitive to outliers, difficult to choose a meaningful query time t). A more flexible and general ranking operation is to rank objects based on the aggregation of their scores in a query interval, which we dub the aggregate top-k query on temporal data. For example, return the top-10 weather stations having the highest average temperature from 10/01/2010 to 10/07/2010; find the top-20 stocks having the largest total transaction volumes from 02/05/2011 to 02/07/2011. This work presents a comprehensive study to this problem by designing both exact and approximate methods (with approximation quality guarantees). We also provide theoretical analysis on the construction cost, the index size, the update and the query costs of each approach. Extensive experiments on large real datasets clearly demonstrate the efficiency, the effectiveness, and the scalability of our methods compared to the baseline methods.


Compacting Transactional Data in Hybrid OLTP & OLAP Databases

Abstract: Growing main memory sizes have facilitated database management systems that keep the entire database in main memory. The drastic performance improvements that came along with these in-memory systems have made it possible to reunite the two areas of online transaction processing (OLTP) and online analytical processing (OLAP): An emerging class of hybrid OLTP and OLAP database systems allows to process analytical queries directly on the transactional data. By offering arbitrarily current snapshots of the transactional data for OLAP, these systems enable real-time business intelligence. Despite memory sizes of several Terabytes in a single commodity server, RAM is still a precious resource: Since free memory can be used for intermediate results in query processing, the amount of memory determines query performance to a large extent. Consequently, we propose the compaction of memory-resident databases. Compaction consists of two tasks: First, separating the mutable working set from the immutable "frozen" data. Second, compressing the immutable data and optimizing it for efficient, memory-consumption-friendly snapshotting. Our approach reorganizes and compresses transactional data online and yet hardly affects the mission-critical OLTP throughput. This is achieved by unburdening the OLTP threads from all additional processing and performing these tasks asynchronously.


Processing a Trillion Cells per Mouse Click

Abstract: Column-oriented database systems have been a real game changer for the industry in recent years. Highly tuned and performant systems have evolved that provide users with the possibility of answering ad hoc queries over large datasets in an interactive manner. In this paper we present the column-oriented datastore developed as one of the central components of PowerDrill. It combines the advantages of columnar data layout with other known techniques (such as using composite range partitions) and extensive algorithmic engineering on key data structures. The main goal of the latter being to reduce the main memory footprint and to increase the efficiency in processing typical user queries. In this combination we achieve large speed-ups. These enable a highly interactive Web UI where it is common that a single mouse click leads to processing a trillion values in the underlying dataset.


OLTP on Hardware Islands

Abstract: Modern hardware is abundantly parallel and increasingly heterogeneous. The numerous processing cores have non-uniform access latencies to the main memory and to the processor caches, which causes variability in the communication costs. Unfortunately, database systems mostly assume that all processing cores are the same and that microarchitecture differences are not significant enough to appear in critical database execution paths. As we demonstrate in this paper, however, hardware heterogeneity does appear in the critical path and conventional database architectures achieve suboptimal and even worse, unpredictable performance. We perform a detailed performance analysis of OLTP deployments in servers with multiple cores per CPU (multicore) and multiple CPUs per server (multisocket). We compare different database deployment strategies where we vary the number and size of independent database instances running on a single server, from a single shared-everything instance to fine-grained shared-nothing configurations. We quantify the impact of non-uniform hardware on various deployments by (a) examining how efficiently each deployment uses the available hardware resources and (b) measuring the impact of distributed transactions and skewed requests on different workloads. Finally, we argue in favor of shared-nothing deployments that are topology- and workload-aware and take advantage of fast on-chip communication between islands of cores on the same socket.


Hamming Approximation of NP Witnesses

Abstract: Given a satisfiable 3-SAT formula, how hard is it to find an assignment to the variables that has Hamming distance at most n/2 to a satisfying assignment? More generally, consider any polynomial-time verifier for any NP-complete language. A d(n)-Hamming-approximation algorithm for the verifier is one that, given any member x of the language, outputs in polynomial time a string a with Hamming distance at most d(n) to some witness w, where (x,w) is accepted by the verifier. Previous results have shown that, if P != NP, then every NP-complete language has a verifier for which there is no (n/2-n^(2/3+d))-Hamming-approximation algorithm, for various constants d > 0. Our main result is that, if P != NP, then every paddable NP-complete language has a verifier that admits no (n/2+O(sqrt(n log n)))-Hamming-approximation algorithm. That is, one cannot get even half the bits right. We also consider natural verifiers for various well-known NP-complete problems. They do have n/2-Hamming-approximation algorithms, but, if P != NP, have no (n/2-n^epsilon)-Hamming-approximation algorithms for any constant epsilon > 0. We show similar results for randomized algorithms.


Electronic administration in Spain: from its beginnings to the present

Abstract: This study presents the basic lines of electronic administration in Spain. The complexity of the Spanish political-administrative system makes such a study challenging, in view of the considerable degree of autonomy and competences of the regional administrative bodies and local agencies with respect to the central government, the former being more visible in the 17 regions of Spain. Nonetheless, the central government maintains a series of legal instruments that allow a certain common framework of action to be imposed, aside from what is put into effect through diverse programs aimed precisely to develop common tools for the regions and municipalities of Spain. After an introduction that provides some necessary background, this study describes the legislative framework in which Spain's electronic administrative system has developed. The data included in the study refer to investment in information and communication technologies (ICT) and the services offered by the different Administrations on the internet; internet access by citizens, homes, businesses, and employees, as well as the interactivity existing with administrations by means of the internet; the origins and rise of various political initiatives of the Central Government involving electronic administration; and finally, the situation of civil service personnel, as catalysts of the success of Information Society in the Public Administration within Spain.


Serializability, not Serial: Concurrency Control and Availability in Multi-Datacenter Datastores

Abstract: We present a framework for concurrency control and availability in multi-datacenter datastores. While we consider Google's Megastore as our motivating example, we define general abstractions for key components, making our solution extensible to any system that satisfies the abstraction properties. We first develop and analyze a transaction management and replication protocol based on a straightforward implementation of the Paxos algorithm. Our investigation reveals that this protocol acts as a concurrency prevention mechanism rather than a concurrency control mechanism. We then propose an enhanced protocol called Paxos with Combination and Promotion (Paxos-CP) that provides true transaction concurrency while requiring the same per instance message complexity as the basic Paxos protocol. Finally, we compare the performance of Paxos and Paxos-CP in a multi-datacenter experimental study, and we demonstrate that Paxos-CP results in significantly fewer aborted transactions than basic Paxos.


Automatic Partitioning of Database Applications

Abstract: Database-backed applications are nearly ubiquitous in our daily lives. Applications that make many small accesses to the database create two challenges for developers: increased latency and wasted resources from numerous network round trips. A well-known technique to improve transactional database application performance is to convert part of the application into stored procedures that are executed on the database server. Unfortunately, this conversion is often difficult. In this paper we describe Pyxis, a system that takes database-backed applications and automatically partitions their code into two pieces, one of which is executed on the application server and the other on the database server. Pyxis profiles the application and server loads, statically analyzes the code's dependencies, and produces a partitioning that minimizes the number of control transfers as well as the amount of data sent during each transfer. Our experiments using TPC-C and TPC-W show that Pyxis is able to generate partitions with up to 3x reduction in latency and 1.7x improvement in throughput when compared to a traditional non-partitioned implementation and has comparable performance to that of a custom stored procedure implementation.


Whom to Ask? Jury Selection for Decision Making Tasks on Micro-blog Services

Abstract: It is universal to see people obtain knowledge on micro-blog services by asking others decision making questions. In this paper, we study the Jury Selection Problem(JSP) by utilizing crowdsourcing for decision making tasks on micro-blog services. Specifically, the problem is to enroll a subset of crowd under a limited budget, whose aggregated wisdom via Majority Voting scheme has the lowest probability of drawing a wrong answer(Jury Error Rate-JER). Due to various individual error-rates of the crowd, the calculation of JER is non-trivial. Firstly, we explicitly state that JER is the probability when the number of wrong jurors is larger than half of the size of a jury. To avoid the exponentially increasing calculation of JER, we propose two efficient algorithms and an effective bounding technique. Furthermore, we study the Jury Selection Problem on two crowdsourcing models, one is for altruistic users(AltrM) and the other is for incentive-requiring users(PayM) who require extra payment when enrolled into a task. For the AltrM model, we prove the monotonicity of JER on individual error rate and propose an efficient exact algorithm for JSP. For the PayM model, we prove the NP-hardness of JSP on PayM and propose an efficient greedy-based heuristic algorithm. Finally, we conduct a series of experiments to investigate the traits of JSP, and validate the efficiency and effectiveness of our proposed algorithms on both synthetic and real micro-blog data.


ALAE: Accelerating Local Alignment with Affine Gap Exactly in Biosequence Databases

Abstract: We study the problem of local alignment, which is finding pairs of similar subsequences with gaps. The problem exists in biosequence databases. BLAST is a typical software for finding local alignment based on heuristic, but could miss results. Using the Smith-Waterman algorithm, we can find all local alignments in O(mn) time, where m and n are lengths of a query and a text, respectively. A recent exact approach BWT-SW improves the complexity of the Smith-Waterman algorithm under constraints, but still much slower than BLAST. This paper takes on the challenge of designing an accurate and efficient algorithm for evaluating local-alignment searches, especially for long queries. In this paper, we propose an efficient software called ALAE to speed up BWT-SW using a compressed suffix array. ALAE utilizes a family of filtering techniques to prune meaningless calculations and an algorithm for reusing score calculations. We also give a mathematical analysis and show that the upper bound of the total number of calculated entries using ALAE could vary from 4.50mn0.520 to 9.05mn0.896 for random DNA sequences and vary from 8.28mn0.364 to 7.49mn0.723 for random protein sequences. We demonstrate the significant performance improvement of ALAE on BWT-SW using a thorough experimental study on real biosequences. ALAE guarantees correctness and accelerates BLAST for most of parameters.


sDTW: Computing DTW Distances using Locally Relevant Constraints based on Salient Feature Alignments

Abstract: Many applications generate and consume temporal data and retrieval of time series is a key processing step in many application domains. Dynamic time warping (DTW) distance between time series of size N and M is computed relying on a dynamic programming approach which creates and fills an NxM grid to search for an optimal warp path. Since this can be costly, various heuristics have been proposed to cut away the potentially unproductive portions of the DTW grid. In this paper, we argue that time series often carry structural features that can be used for identifying locally relevant constraints to eliminate redundant work. Relying on this observation, we propose salient feature based sDTW algorithms which first identify robust salient features in the given time series and then find a consistent alignment of these to establish the boundaries for the warp path search. More specifically, we propose alternative fixed core&adaptive width, adaptive core&fixed width, and adaptive core&adaptive width strategies which enforce different constraints reflecting the high level structural characteristics of the series in the data set. Experiment results show that the proposed sDTW algorithms help achieve much higher accuracy in DTWcomputation and time series retrieval than fixed core & fixed width algorithms that do not leverage local features of the given time series.


SCOUT: Prefetching for Latent Feature Following Queries

Abstract: Today's scientists are quickly moving from in vitro to in silico experimentation: they no longer analyze natural phenomena in a petri dish, but instead they build models and simulate them. Managing and analyzing the massive amounts of data involved in simulations is a major task. Yet, they lack the tools to efficiently work with data of this size. One problem many scientists share is the analysis of the massive spatial models they build. For several types of analysis they need to interactively follow the structures in the spatial model, e.g., the arterial tree, neuron fibers, etc., and issue range queries along the way. Each query takes long to execute, and the total time for executing a sequence of queries significantly delays data analysis. Prefetching the spatial data reduces the response time considerably, but known approaches do not prefetch with high accuracy. We develop SCOUT, a structure-aware method for prefetching data along interactive spatial query sequences. SCOUT uses an approximate graph model of the structures involved in past queries and attempts to identify what particular structure the user follows. Our experiments with neuroscience data show that SCOUT prefetches with an accuracy from 71% to 92%, which translates to a speedup of 4x-15x. SCOUT also improves the prefetching accuracy on datasets from other scientific domains, such as medicine and biology.


Accelerating Pathology Image Data Cross-Comparison on CPU-GPU Hybrid Systems

Abstract: As an important application of spatial databases in pathology imaging analysis, cross-comparing the spatial boundaries of a huge amount of segmented micro-anatomic objects demands extremely data- and compute-intensive operations, requiring high throughput at an affordable cost. However, the performance of spatial database systems has not been satisfactory since their implementations of spatial operations cannot fully utilize the power of modern parallel hardware. In this paper, we provide a customized software solution that exploits GPUs and multi-core CPUs to accelerate spatial cross-comparison in a cost-effective way. Our solution consists of an efficient GPU algorithm and a pipelined system framework with task migration support. Extensive experiments with real-world data sets demonstrate the effectiveness of our solution, which improves the performance of spatial cross-comparison by over 18 times compared with a parallelized spatial database approach.


Robust Estimation of Resource Consumption for SQL Queries using Statistical Techniques

Abstract: The ability to estimate resource consumption of SQL queries is crucial for a number of tasks in a database system such as admission control, query scheduling and costing during query optimization. Recent work has explored the use of statistical techniques for resource estimation in place of the manually constructed cost models used in query optimization. Such techniques, which require as training data examples of resource usage in queries, offer the promise of superior estimation accuracy since they can account for factors such as hardware characteristics of the system or bias in cardinality estimates. However, the proposed approaches lack robustness in that they do not generalize well to queries that are different from the training examples, resulting in significant estimation errors. Our approach aims to address this problem by combining knowledge of database query processing with statistical models. We model resource-usage at the level of individual operators, with different models and features for each operator type, and explicitly model the asymptotic behavior of each operator. This results in significantly better estimation accuracy and the ability to estimate resource usage of arbitrary plans, even when they are very different from the training instances. We validate our approach using various large scale real-life and benchmark workloads on Microsoft SQL Server.


A Parametric Worst-Case Approach to Fairness in TU-Cooperative Games

Abstract: We propose a parametric family of measures of fairness in allocations of TU-cooperative games. Their definition is based on generalized Renyi Entropy, is related to the Cowell-Kuga generalized entropy indices in welfare economics, and aims to parallel the spirit of the notion of price of anarchy in the case of convex TU-cooperative games. Since computing these indices is NP-complete in general, we first upper bound the performance of a "reverse greedy" algorithm for approximately computing worst-case fairness. The result provides a general additive error guarantee in terms of two (problem dependent) packing constants. We then particularize this result to the class of induced subset games. For such games computing worst-case fairness is NP-complete, and the additive guarantee constant can be explicitly computed. We compare this result to the performance of an alternate algorithm based on "biased orientations".


Who Tags What? An Analysis Framework

Abstract: The rise of Web 2.0 is signaled by sites such as Flickr, del.icio.us, and YouTube, and social tagging is essential to their success. A typical tagging action involves three components, user, item (e.g., photos in Flickr), and tags (i.e., words or phrases). Analyzing how tags are assigned by certain users to certain items has important implications in helping users search for desired information. In this paper, we explore common analysis tasks and propose a dual mining framework for social tagging behavior mining. This framework is centered around two opposing measures, similarity and diversity, being applied to one or more tagging components, and therefore enables a wide range of analysis scenarios such as characterizing similar users tagging diverse items with similar tags, or diverse users tagging similar items with diverse tags, etc. By adopting different concrete measures for similarity and diversity in the framework, we show that a wide range of concrete analysis problems can be defined and they are NP-Complete in general. We design efficient algorithms for solving many of those problems and demonstrate, through comprehensive experiments over real data, that our algorithms significantly out-perform the exact brute-force approach without compromising analysis result quality.


A Generic Framework for Efficient and Effective Subsequence Retrieval

Abstract: This paper proposes a general framework for matching similar subsequences in both time series and string databases. The matching results are pairs of query subsequences and database subsequences. The framework finds all possible pairs of similar subsequences if the distance measure satisfies the "consistency" property, which is a property introduced in this paper. We show that most popular distance functions, such as the Euclidean distance, DTW, ERP, the Frechet distance for time series, and the Hamming distance and Levenshtein distance for strings, are all "consistent". We also propose a generic index structure for metric spaces named "reference net". The reference net occupies O(n) space, where n is the size of the dataset and is optimized to work well with our framework. The experiments demonstrate the ability of our method to improve retrieval performance when combined with diverse distance measures. The experiments also illustrate that the reference net scales well in terms of space overhead and query time.


Only Aggressive Elephants are Fast Elephants

Abstract: Yellow elephants are slow. A major reason is that they consume their inputs entirely before responding to an elephant rider's orders. Some clever riders have trained their yellow elephants to only consume parts of the inputs before responding. However, the teaching time to make an elephant do that is high. So high that the teaching lessons often do not pay off. We take a different approach. We make elephants aggressive; only this will make them very fast. We propose HAIL (Hadoop Aggressive Indexing Library), an enhancement of HDFS and Hadoop MapReduce that dramatically improves runtimes of several classes of MapReduce jobs. HAIL changes the upload pipeline of HDFS in order to create different clustered indexes on each data block replica. An interesting feature of HAIL is that we typically create a win-win situation: we improve both data upload to HDFS and the runtime of the actual Hadoop MapReduce job. In terms of data upload, HAIL improves over HDFS by up to 60% with the default replication factor of three. In terms of query execution, we demonstrate that HAIL runs up to 68x faster than Hadoop. In our experiments, we use six clusters including physical and EC2 clusters of up to 100 nodes. A series of scalability experiments also demonstrates the superiority of HAIL.


Multiple Location Profiling for Users and Relationships from Social Network and Content

Abstract: Users' locations are important for many applications such as personalized search and localized content delivery. In this paper, we study the problem of profiling Twitter users' locations with their following network and tweets. We propose a multiple location profiling model (MLP), which has three key features: 1) it formally models how likely a user follows another user given their locations and how likely a user tweets a venue given his location, 2) it fundamentally captures that a user has multiple locations and his following relationships and tweeted venues can be related to any of his locations, and some of them are even noisy, and 3) it novelly utilizes the home locations of some users as partial supervision. As a result, MLP not only discovers users' locations accurately and completely, but also "explains" each following relationship by revealing users' true locations in the relationship. Experiments on a large-scale data set demonstrate those advantages. Particularly, 1) for predicting users' home locations, MLP successfully places 62% users and outperforms two state-of-the-art methods by 10% in accuracy, 2) for discovering users' multiple locations, MLP improves the baseline methods by 14% in recall, and 3) for explaining following relationships, MLP achieves 57% accuracy.


Flash-based Extended Cache for Higher Throughput and Faster Recovery

Abstract: Considering the current price gap between disk and flash memory drives, for applications dealing with large scale data, it will be economically more sensible to use flash memory drives to supplement disk drives rather than to replace them. This paper presents FaCE, which is a new low-overhead caching strategy that uses flash memory as an extension to the DRAM buffer. FaCE aims at improving the transaction throughput as well as shortening the recovery time from a system failure. To achieve the goals, we propose two novel algorithms for flash cache management, namely, Multi-Version FIFO replacement and Group Second Chance. One striking result from FaCE is that using a small flash memory drive as a caching device could deliver even higher throughput than using a large flash memory drive to store the entire database tables. This was possible due to flash write optimization as well as disk access reduction obtained by the FaCE caching methods. In addition, FaCE takes advantage of the non-volatility of flash memory to fully support database recovery by extending the scope of a persistent database to include the data pages stored in the flash cache. We have implemented FaCE in the PostgreSQL open source database server and demonstrated its effectiveness for TPC-C benchmarks.


Don't Thrash: How to Cache Your Hash on Flash

Abstract: This paper presents new alternatives to the well-known Bloom filter data structure. The Bloom filter, a compact data structure supporting set insertion and membership queries, has found wide application in databases, storage systems, and networks. Because the Bloom filter performs frequent random reads and writes, it is used almost exclusively in RAM, limiting the size of the sets it can represent. This paper first describes the quotient filter, which supports the basic operations of the Bloom filter, achieving roughly comparable performance in terms of space and time, but with better data locality. Operations on the quotient filter require only a small number of contiguous accesses. The quotient filter has other advantages over the Bloom filter: it supports deletions, it can be dynamically resized, and two quotient filters can be efficiently merged. The paper then gives two data structures, the buffered quotient filter and the cascade filter, which exploit the quotient filter advantages and thus serve as SSD-optimized alternatives to the Bloom filter. The cascade filter has better asymptotic I/O performance than the buffered quotient filter, but the buffered quotient filter outperforms the cascade filter on small to medium data sets. Both data structures significantly outperform recently-proposed SSD-optimized Bloom filter variants, such as the elevator Bloom filter, buffered Bloom filter, and forest-structured Bloom filter. In experiments, the cascade filter and buffered quotient filter performed insertions 8.6-11 times faster than the fastest Bloom filter variant and performed lookups 0.94-2.56 times faster.


Learning Expressive Linkage Rules using Genetic Programming

Abstract: A central problem in data integration and data cleansing is to find entities in different data sources that describe the same real-world object. Many existing methods for identifying such entities rely on explicit linkage rules which specify the conditions that entities must fulfill in order to be considered to describe the same real-world object. In this paper, we present the GenLink algorithm for learning expressive linkage rules from a set of existing reference links using genetic programming. The algorithm is capable of generating linkage rules which select discriminative properties for comparison, apply chains of data transformations to normalize property values, choose appropriate distance measures and thresholds and combine the results of multiple comparisons using non-linear aggregation functions. Our experiments show that the GenLink algorithm outperforms the state-of-the-art genetic programming approach to learning linkage rules recently presented by Carvalho et. al. and is capable of learning linkage rules which achieve a similar accuracy as human written rules for the same problem.


Mining Frequent Itemsets over Uncertain Databases

Abstract: In recent years, due to the wide applications of uncertain data, mining frequent itemsets over uncertain databases has attracted much attention. In uncertain databases, the support of an itemset is a random variable instead of a fixed occurrence counting of this itemset. Thus, unlike the corresponding problem in deterministic databases where the frequent itemset has a unique definition, the frequent itemset under uncertain environments has two different definitions so far. The first definition, referred as the expected support-based frequent itemset, employs the expectation of the support of an itemset to measure whether this itemset is frequent. The second definition, referred as the probabilistic frequent itemset, uses the probability of the support of an itemset to measure its frequency. Thus, existing work on mining frequent itemsets over uncertain databases is divided into two different groups and no study is conducted to comprehensively compare the two different definitions. In addition, since no uniform experimental platform exists, current solutions for the same definition even generate inconsistent results. In this paper, we firstly aim to clarify the relationship between the two different definitions. Through extensive experiments, we verify that the two definitions have a tight connection and can be unified together when the size of data is large enough. Secondly, we provide baseline implementations of eight existing representative algorithms and test their performances with uniform measures fairly. Finally, according to the fair tests over many different benchmark data sets, we clarify several existing inconsistent conclusions and discuss some new findings.


The Distributed Ontology Language (DOL): Use Cases, Syntax, and Extensibility

Abstract: The Distributed Ontology Language (DOL) is currently being standardized within the OntoIOp (Ontology Integration and Interoperability) activity of ISO/TC 37/SC 3. It aims at providing a unified framework for (1) ontologies formalized in heterogeneous logics, (2) modular ontologies, (3) links between ontologies, and (4) annotation of ontologies. This paper presents the current state of DOL's standardization. It focuses on use cases where distributed ontologies enable interoperability and reusability. We demonstrate relevant features of the DOL syntax and semantics and explain how these integrate into existing knowledge engineering environments.


Equilibria of Chinese Auctions

Abstract: Chinese auctions are a combination between a raffle and an auction and are held in practice at charity events or festivals. In a Chinese auction, multiple players compete for several items by buying tickets, which can be used to win the items. In front of each item there is a basket, and the players can bid by placing tickets in the basket(s) corresponding to the item(s) they are trying to win. After all the players have placed their tickets, a ticket is drawn at random from each basket and the item is given to the owner of the winning ticket. While a player is never guaranteed to win an item, they can improve their chances of getting it by increasing the number of tickets for that item. In this paper we investigate the existence of pure Nash equilibria in both the continuous and discrete settings. When the players have continuous budgets, we show that a pure Nash equilibrium may not exist for asymmetric games when some valuations are zero. In that case we prove that the auctioneer can stabilize the game by placing his own ticket in each basket. On the other hand, when all the valuations are strictly positive, a pure Nash equilibrium is guaranteed to exist, and the equilibrium strategies are symmetric when both valuations and budgets are symmetric. We also study Chinese auctions with discrete budgets, for which we give both existence results and counterexamples. While the literature on rent-seeking contests traditionally focuses on continuous costly tickets, the discrete variant is very natural and more closely models the version of the auction held in practice.


A Model for Minimizing Active Processor Time

Abstract: We introduce the following elementary scheduling problem. We are given a collection of n jobs, where each job has an integer length as well as a set Ti of time intervals in which it can be feasibly scheduled. Given a parameter B, the processor can schedule up to B jobs at a timeslot t so long as it is "active" at t. The goal is to schedule all the jobs in the fewest number of active timeslots. The machine consumes a fixed amount of energy per active timeslot, regardless of the number of jobs scheduled in that slot (as long as the number of jobs is non-zero). In other words, subject to all units of each job being scheduled in its feasible region and at each slot at most B jobs being scheduled, we are interested in minimizing the total time during which the machine is active. We present a linear time algorithm for the case where jobs are unit length and each Ti is a single interval. For general Ti, we show that the problem is NP-complete even for B = 3. However when B = 2, we show that it can be efficiently solved. In addition, we consider a version of the problem where jobs have arbitrary lengths and can be preempted at any point in time. For general B, the problem can be solved by linear programming. For B = 2, the problem amounts to finding a triangle-free 2-matching on a special graph. We extend the algorithm of Babenko et. al. to handle our variant, and also to handle non-unit length jobs. This yields an O(sqrt(L)m) time algorithm to solve the preemptive scheduling problem for B = 2, where L is the sum of the job lengths. We also show that for B = 2 and unit length jobs, the optimal non-preemptive schedule has at most 4/3 times the active time of the optimal preemptive schedule; this bound extends to several versions of the problem when jobs have arbitrary length.


Artificial Neural Network Based Prediction of Optimal Pseudo-Damping and Meta-Damping in Oscillatory Fractional Order Dynamical Systems

Abstract: This paper investigates typical behaviors like damped oscillations in fractional order (FO) dynamical systems. Such response occurs due to the presence of, what is conceived as, pseudo-damping and meta-damping in some special class of FO systems. Here, approximation of such damped oscillation in FO systems with the conventional notion of integer order damping and time constant has been carried out using Genetic Algorithm (GA). Next, a multilayer feed-forward Artificial Neural Network (ANN) has been trained using the GA based results to predict the optimal pseudo and meta-damping from knowledge of the maximum order or number of terms in the FO dynamical system.


An Automat for the Semantic Processing of Structured Information

Abstract: Using the database of the PuertoTerm project, an indexing system based on the cognitive model of Brigitte Enders was built. By analyzing the cognitive strategies of three abstractors, we built an automat that serves to simulate human indexing processes. The automat allows the texts integrated in the system to be assessed, evaluated and grouped by means of the bipartite spectral graph partitioning algorithm, which also permits visualization of the terms and the documents. The system features an ontology and a database to enhance its operativity. As a result of the application, we achieved better rates of exhaustivity in the indexing of documents, as well as greater precision and retrieval of information, with high levels of efficiency.


An Economic Analysis of User-Privacy Options in Ad-Supported Services

Abstract: We analyze the value to e-commerce website operators of offering privacy options to users, e.g., of allowing users to opt out of ad targeting. In particular, we assume that site operators have some control over the cost that a privacy option imposes on users and ask when it is to their advantage to make such costs low. We consider both the case of a single site and the case of multiple sites that compete both for users who value privacy highly and for users who value it less. One of our main results in the case of a single site is that, under normally distributed utilities, if a privacy-sensitive user is worth at least $\sqrt{2} - 1$ times as much to advertisers as a privacy-insensitive user, the site operator should strive to make the cost of a privacy option as low as possible. In the case of multiple sites, we show how a Prisoner's-Dilemma situation can arise: In the equilibrium in which both sites are obliged to offer a privacy option at minimal cost, both sites obtain lower revenue than they would if they colluded and neither offered a privacy option.


Global Adaptive Routing Algorithm Without Additional Congestion Propagation Network

Abstract: Adaptive routing algorithm has been employed in multichip interconnection networks in order to improve network performance. Does a algorithm use local or global network state? This is the key question in adaptive routing. In many traffic patterns, the ignorance of global network state, leading to routing selection based only on local congestion information, tends to violate global load balance. To attack the load balance issue in adapting routing, some global adaptive routing algorithms introduce a congestion propagation network to obtain global network status information, such as Regional Congestion Awareness (RCA) and Destination Based Adaptive Routing (DBAR). However, the congestion propagation network leads to additional power and area consumption which cannot be ignored. From another view, if we just increase the bandwidth between neighbor nodes with the wires used to build the congestion propagation network, the network performance could be improved as well. In this paper, we propose a global adaptive routing algorithm without employing the additional congestion propagation network. Our algorithm obtains the global network state in a novel way, and can offer significant improvement than the base-line local adaptive routing algorithm (xy-adaptive algorithm which selects routing based on local congestion information in each hop) for both medium and high injection rates. In wormhole flow control, all the routing information (flit id, source node id, destination node id, vc id and address) is contained in head flit, and data is carried in body flits. As a result, there are always many free bits in the head flit, especially when the bandwidth is 128-bits which is normal in interconnection network design. Then, we can use these free bits in the head flit to propagate global congestion information but not increase the number of flits.


Linear Time Algorithm for Optimal Feed-link Placement

Abstract: Given a polygon representing a transportation network together with a point p in its interior, we aim to extend the network by inserting a line segment, called a feed-link, which connects p to the boundary of the polygon. Once a feed link is fixed, the geometric dilation of some point q on the boundary is the ratio between the length of the shortest path from p to q through the extended network, and their Euclidean distance. The utility of a feed-link is inversely proportional to the maximal dilation over all boundary points. We give a linear time algorithm for computing the feed-link with the minimum overall dilation, thus improving upon the previously known algorithm of complexity that is roughly O(n log n).


Solving Cyclic Longest Common Subsequence in Quadratic Time

Abstract: We present a practical algorithm for the cyclic longest common subsequence (CLCS) problem that runs in O(mn) time, where m and n are the lengths of the two input strings. While this is not necessarily an asymptotic improvement over the existing record, it is far simpler to understand and to implement.


Local public good provisioning in networks: A Nash implementation mechanism

Abstract: In this paper we study resource allocation in decentralized information local public good networks. A network is a local public good network if each user's actions directly affect the utility of an arbitrary subset of network users. We consider networks where each user knows only that part of the network that either affects or is affected by it. Furthermore, each user's utility and action space are its private information, and each user is a self utility maximizer. This network model is motivated by several applications including wireless communications and online advertising. For this network model we formulate a decentralized resource allocation problem and develop a decentralized resource allocation mechanism (game form) that possesses the following properties: (i) All Nash equilibria of the game induced by the mechanism result in allocations that are optimal solutions of the corresponding centralized resource allocation problem (Nash implementation). (ii) All users voluntarily participate in the allocation process specified by the mechanism (individual rationality). (iii) The mechanism results in budget balance at all Nash equilibria and off equilibrium.


Botnet-based Distributed Denial of Service (DDoS) Attacks on Web Servers: Classification and Art

Abstract: Botnets are prevailing mechanisms for the facilitation of the distributed denial of service (DDoS) attacks on computer networks or applications. Currently, Botnet-based DDoS attacks on the application layer are latest and most problematic trends in network security threats. Botnet-based DDoS attacks on the application layer limits resources, curtails revenue, and yields customer dissatisfaction, among others. DDoS attacks are among the most difficult problems to resolve online, especially, when the target is the Web server. In this paper, we present a comprehensive study to show the danger of Botnet-based DDoS attacks on application layer, especially on the Web server and the increased incidents of such attacks that has evidently increased recently. Botnet-based DDoS attacks incidents and revenue losses of famous companies and government websites are also described. This provides better understanding of the problem, current solution space, and future research scope to defend against such attacks efficiently.


True-MCSA: A Framework for Truthful Double Multi-Channel Spectrum Auctions

Abstract: We design a framework for truthful double multi-channel spectrum auctions where each seller (or buyer) can sell (or buy) multiple spectrum channels based on their individual needs. Open, market-based spectrum trading motivates existing spectrum owners (as sellers) to lease their selected idle spectrum channels to new spectrum users (as buyers) who need the spectrum desperately. The most significant requirement is how to make the auction economic-robust (truthful in particular) while enabling spectrum reuse to improve spectrum utilization. Additionally, in practice, both sellers and buyers would require to trade multiple channels at one time, while guaranteeing their individual profitability. Unfortunately, none of the existing designs can meet all these requirements simultaneously. We address these requirements by proposing True-MCSA, a framework for truthful double multi-channel spectrum auctions. True-MCSA takes as input any reusability-driven spectrum allocation algorithm, introduces novel virtual buyer group (VBG) splitting and bidding algorithms, and applies a winner determination and pricing mechanism to achieve truthfulness and other economic properties while improving spectrum utilization and successfully dealing with multi-channel requests from both buyers and sellers. Our results show that the auction efficiency is impacted by the economic factors with efficiency degradations within 30%, under different experimental settings. Furthermore, the experimental results indicate that we can improve the auction efficiency by choosing a proper bidding algorithm and using a base bid. True-MCSA makes an important contribution on enabling spectrum reuse to improve auction efficiency in multi-channel cases.


Fixed Interfaces, Adaptive Interfaces... What is next? Total movability - a new paradigm for the user interface

Abstract: Users can't talk with computers in their natural language (machine codes), so there are interfaces that allow such communication. 40 years ago the outcome of computer programs was in the form of long listings covered by numbers and even the format of those numbers was determined by developers. Throughout the latest 25 years: program views and results are shown in a wide variety of shapes and variants, but all these possibilities are predefined and fixed in code by developers; nothing outside of their approved solutions is allowed. My vision from now on into the future: developers are responsible only for correct work of a program (calculations, link with the database, etc.) and suggest a good default interface, but not determine all possible scenarios; only users decide WHAT, WHEN, and HOW to show. This will be a revolution in our dealing with computers, but there are obvious questions. How this step can be made? Do all users need such change? Is it going to be a burden for users or a welcome revolution?


Rejecting the Attack: Source Authentication for Wi-Fi Management Frames using CSI Information

Abstract: Comparing to well protected data frames, Wi-Fi management frames (MFs) are extremely vulnerable to various attacks. Since MFs are transmitted without encryption, attackers can forge them easily. Such attacks can be detected in cooperative environment such as Wireless Intrusion Detection System (WIDS). However, in non-cooperative environment it is difficult for a single station to identify these spoofing attacks using Received Signal Strength (RSS)-based detection, due to the strong correlation of RSS to both the transmission power (Txpower) and the location of the sender. By exploiting some unique characteristics (i.e., rapid spatial decorrelation, independence of Txpower, and much richer dimensions) of the Channel State Information (CSI), a standard feature in 802.11n Specification, we design a prototype, called CSITE, to authenticate the Wi-Fi management frames by a single station without external support. Our design CSITE, built upon off-the-shelf hardware, achieves precise spoofing detection without collaboration and in-advance finger-print. Several novel techniques are designed to address the challenges caused by user mobility and channel dynamics. To verify the performances of our solution, we implement a prototype of our design and conduct extensive evaluations in various scenarios. Our test results show that our design significantly outperforms the RSS-based method in terms of accuracy, robustness, and efficiency: we observe about 8 times improvement by CSITE over RSS-based method on the falsely accepted attacking frames.


Diamond-free Degree Sequences

Abstract: We introduce a new problem, CSPLib problem number 50, to generate all degree sequences that have a corresponding diamond-free graph with secondary properties. This problem arises naturally from a problem in mathematics to do with balanced incomplete block designs; we devote a section of this paper to this. The problem itself is challenging with respect to computational effort arising from the large number of symmetries within the models. We introduce two models for this problem. The second model is an improvement on the first, and this improvement largely consists of breaking the problem into two stages, the first stage producing graphical degree sequences that satisfy arithmetic constraints and the second part testing that there exists a graph with that degree sequence that is diamond-free.


Criticality of Large Delay Tolerant Networks via Directed Continuum Percolation in Space-Time

Abstract: We study delay tolerant networking (DTN) and in particular, its capacity to store, carry and forward messages so that the messages eventually reach their final destinations. We approach this broad question in the framework of percolation theory. To this end, we assume an elementary mobility model, where nodes arrive to an infinite plane according to a Poisson point process, move a certain distance L, and then depart. In this setting, we characterize the mean density of nodes required to support DTN style networking. In particular, under the given assumptions, we show that DTN is feasible when the mean node degree is greater than 4 e(g), where parameter g=L/d is the ratio of the distance L to the transmission range d, and e(g) is the critical reduced number density of tilted cylinders in a directed continuum percolation model. By means of Monte Carlo simulations, we give numerical values for e(g). The asymptotic behavior of e(g) when g tends to infinity is also derived from a fluid flow analysis.


On Constructor Rewrite Systems and the Lambda Calculus

Abstract: We prove that orthogonal constructor term rewrite systems and lambda-calculus with weak (i.e., no reduction is allowed under the scope of a lambda-abstraction) call-by-value reduction can simulate each other with a linear overhead. In particular, weak call-by- value beta-reduction can be simulated by an orthogonal constructor term rewrite system in the same number of reduction steps. Conversely, each reduction in a term rewrite system can be simulated by a constant number of beta-reduction steps. This is relevant to implicit computational complexity, because the number of beta steps to normal form is polynomially related to the actual cost (that is, as performed on a Turing machine) of normalization, under weak call-by-value reduction. Orthogonal constructor term rewrite systems and lambda-calculus are thus both polynomially related to Turing machines, taking as notion of cost their natural parameters.


An Upper Bound on the Convergence Time for Distributed Binary Consensus

Abstract: The problem addressed in this paper is the analysis of a distributed consensus algorithm for arbitrary networks, proposed by B\'en\'ezit et al.. In the initial setting, each node in the network has one of two possible states ("yes" or "no"). Nodes can update their states by communicating with their neighbors via a 2-bit message in an asynchronous clock setting. Eventually, all nodes reach consensus on the majority states. We use the theory of electric networks, random walks, and couplings of Markov chains to derive an O(N4 logN) upper bound for the expected convergence time on an arbitrary graph of size N.


Modular Type-Safety Proofs using Dependant Types

Abstract: While methods of code abstraction and reuse are widespread and well researched, methods of proof abstraction and reuse are still emerging. We consider the use of dependent types for this purpose, introducing a completely mechanical approach to proof composition. We show that common techniques for abstracting algorithms over data structures naturally translate to abstractions over proofs. We first introduce a language composed of a series of smaller language components tied together by standard techniques from Malcom (1990). We proceed by giving proofs of type preservation for each language component and show that the basic ideas used in composing the syntactic data structures can be applied to their semantics as well.


A hybrid artificial immune system and Self Organising Map for network intrusion detection

Abstract: Network intrusion detection is the problem of detecting unauthorised use of, or access to, computer systems over a network. Two broad approaches exist to tackle this problem: anomaly detection and misuse detection. An anomaly detection system is trained only on examples of normal connections, and thus has the potential to detect novel attacks. However, many anomaly detection systems simply report the anomalous activity, rather than analysing it further in order to report higher-level information that is of more use to a security officer. On the other hand, misuse detection systems recognise known attack patterns, thereby allowing them to provide more detailed information about an intrusion. However, such systems cannot detect novel attacks. A hybrid system is presented in this paper with the aim of combining the advantages of both approaches. Specifically, anomalous network connections are initially detected using an artificial immune system. Connections that are flagged as anomalous are then categorised using a Kohonen Self Organising Map, allowing higher-level information, in the form of cluster membership, to be extracted. Experimental results on the KDD 1999 Cup dataset show a low false positive rate and a detection and classification rate for Denial-of-Service and User-to-Root attacks that is higher than those in a sample of other works.


A Constructive Algorithm to Prove P=NP

Abstract: After reducing the undirected Hamiltonian cycle problem into the TSP problem with cost 0 or 1, we developed an effective algorithm to compute the optimal tour of the transformed TSP. Our algorithm is described as a growth process: initially, constructing 4-vertexes optimal tour; next, one new vertex being added into the optimal tour in such a way to obtain the new optimal tour; then, repeating the previous step until all vertexes are included into the optimal tour. This paper has shown that our constructive algorithm can solve the undirected Hamiltonian cycle problem in polynomial time. According to Cook-Levin theorem, we argue that we have provided a constructive proof of P=NP.


Fast Monotone Summation over Disjoint Sets

Abstract: We study the problem of computing an ensemble of multiple sums where the summands in each sum are indexed by subsets of size $p$ of an $n$-element ground set. More precisely, the task is to compute, for each subset of size $q$ of the ground set, the sum over the values of all subsets of size $p$ that are disjoint from the subset of size $q$. We present an arithmetic circuit that, without subtraction, solves the problem using $O((n^p+n^q)\log n)$ arithmetic gates, all monotone; for constant $p$, $q$ this is within the factor $\log n$ of the optimal. The circuit design is based on viewing the summation as a "set nucleation" task and using a tree-projection approach to implement the nucleation. Applications include improved algorithms for counting heaviest $k$-paths in a weighted graph, computing permanents of rectangular matrices, and dynamic feature selection in machine learning.


Detection of Deviations in Mobile Applications Network Behavior

Abstract: In this paper a novel system for detecting meaningful deviations in a mobile application's network behavior is proposed. The main goal of the proposed system is to protect mobile device users and cellular infrastructure companies from malicious applications. The new system is capable of: (1) identifying malicious attacks or masquerading applications installed on a mobile device, and (2) identifying republishing of popular applications injected with a malicious code. The detection is performed based on the application's network traffic patterns only. For each application two types of models are learned. The first model, local, represents the personal traffic pattern for each user using an application and is learned on the device. The second model, collaborative, represents traffic patterns of numerous users using an application and is learned on the system server. Machine-learning methods are used for learning and detection purposes. This paper focuses on methods utilized for local (i.e., on mobile device) learning and detection of deviations from the normal application's behavior. These methods were implemented and evaluated on Android devices. The evaluation experiments demonstrate that: (1) various applications have specific network traffic patterns and certain application categories can be distinguishable by their network patterns, (2) different levels of deviations from normal behavior can be detected accurately, and (3) local learning is feasible and has a low performance overhead on mobile devices.


Simulation Study For Performance Comparison in Hierarchical Network With CHG Approach in MANET

Abstract: The implementation of MANET for commercial purposes is not an easy task. Unlike other wireless technologies such as cellular networks, MANET face more difficult problems concerning management functions, routing and scalability . As a solution to these complications, clustering schemes are proposed for MANET in order to organize the network topology in a hierarchical manner. Many clustering techniques have been developed. Clustering is a method which aggregates nodes into groups. These groups are contained by the network and they are known as clusters. By Increasing network capacity and reducing the routing overhead through clustering brings more efficiency and effectiveness to scalability in relation to node numbers and the necessity for high mobility. The manager node in clustering has responsibility for many functions such as cluster maintenance, routing table updates, and the discovery of new routes within the network. The other node named as gateway node communicate to the other cluster. In this paper we remove the cluster head (CH) and given a new approach in which cluster head and gateway will be same and that node is known as cluster head gateway (CHG), in which all the responsibilities of cluster head and gateway will be perform by the Cluster head gateway(CHG) itself. By applying this approach we reduce of overheads and improve the over all performance of the network while throughput will be same in both condition with the help of Exata simulation.


Green Telecom Metrics in Perspective

Abstract: A fast and parallel evolution of ways to measure and assess energy efficiency in telecom has resulted in an entangled web of drafts and recommendations originating from government, research, and standards organizations. This paper focuses primarily on so-called "large network equipment" metrics and intends to capture state-of-the-art in this area of green communications. Competing approaches towards efficiency assessment are studied for their applicability and completeness, with special emphasis on topics relevant to future subject studies


Optimal Degree of Optical Circuit Switching in IP-over-WDM Networks

Abstract: In this paper, we study the influence of technology, traffic properties and price trends on optimized design of a reference IP-over-WDM network with rich underlying fiber topology. In each network node, we investigate the optimal degree of traffic switching in an optical (lambda) domain versus an electrical (packet) domain, also known as measure of node transparency. This measure is studied in connection to changes in traffic volume, demand affinity, optical circuit speeds and equipment cost. By applying variable design constraints, we assess the relative roles of the two distinct equipment groups, IP routers and optical cross-connects, with respect to resulting changes in cost-sensitive network architectures.


Translation of Bengali Terms in Mobile Phones: a Simplified Approach Based on the Prescriptions of Conventional Accent Understand Ability

Abstract: Technology is the making, usage and knowledge of tools, techniques, crafts, systems or methods of organization in order to solve a problem or serve some purpose. This is true for humanitarian issues also. Such as the issue of language and its primitive attraction for its native speakers which is visible in the cases of the language spoken at home, outside home, in its choice of newspapers, and TV channels. Everyone finds to accomplish its need by the same way. Example includes the preference of using mobile phones in English. The satisfactory answer to this tendency may be the lack of finding the translations in native language---Bengali terms used in current mobile phones are hard to understand by users. I have investigated various mobile phone models available in Indian market which have lot of problems in Bengali interpretation. I have sort out the root cause of this problem to be the conventional accent understand ability. Depending on this I have created a set of equivalent terms that I hope to be simpler in use. In this paper I have performed experiments to compare the new terms to the available ones. Our findings show that the newly derived terms do better in term of performance than to current ones. It has also been seen that acceptance of Bengali terms in mobile phones might grow if the parameter of simpler and conventional accent understand ability are met while designing.


Hatch-Sens: a Theoretical Bio-Inspired Model to Monitor the Hatching of Plankton Culture in the Vicinity of Wireless Sensor Network

Abstract: Plankton research has always been an important area of biology. Due to various environmental issues and other research interests, plankton hatching and harnessing has been extremely red-marked zone for bio-aqua scientists recently. To counter this problem, no wireless sensor assisted technique or mechanism has yet not been devised. In this literature, we propose a novel approach to pursue this task by the virtue of a theoretical Bio-inspired model named Hatch-Sens, to automatically monitor different parameters of plankton hatching in laboratory environment. This literature illustrates the concepts and detailed mechanisms to accumulate this given problem. Hatch-Sens is a novel idea which combines the biology with computer in its sensing network to monitor hatching parameters of Artemia salina. This model reduces the manual tiresome monitoring of hatching of plankton culture by wireless sensor network.


Debugging Invariant Issues in Pseudo Embedded Program: an Analytical Approach

Abstract: Debugging is an unavoidable and most crucial aspect of software development life cycle. Especially when it comes the turn of embedded one. Due to the requirements of low code size and less resource consumption, the embedded softwares need to be upgraded all the time involving obvious change of code during development phase. This leads the huge risk of intrusion of bugs into the code at production time. In this paper we propose an approach of debugging embedded program in pseudo format, incorporating invariant analysis. Our methodology works on top of Daikon, a popular invariant analyzer. We have experimented with a simplified code snippet [1], used during debugging a reported error in BusyBox which is a de-facto standard for Linux in embedded systems.


The green grid saga - a green initiative to data centers: a review

Abstract: Information Technology (IT) significantly impacts the environment throughout its life cycle. Most enterprises have not paid enough attention to this until recently. IT's environmental impact can be significantly reduced by behavioral changes, as well as technology changes. Given the relative energy and materials inefficiency of most IT infrastructures today, many green IT initiatives can be easily tackled at no incremental cost. The Green Grid - a non-profit trade organization of IT professionals is such an initiative, formed to initiate the issues of power and cooling in data centers, scattered world-wide. The Green Grid seeks to define best practices for optimizing the efficient consumption of power at IT equipment and facility levels, as well as the manner in which cooling is delivered at these levels hence, providing promising attitude in bringing down the environmental hazards, as well as proceeding to the new era of green computing. In this paper we review the various analytical aspects of The Green Grid upon the data centers and found green facts.


Debugging Memory Issues In Embedded Linux: A Case Study

Abstract: Debugging denotes the process of detecting root causes of unexpected observable behaviors in programs, such as a program crash, an unexpected output value being produced or an assertion violation. Debugging of program errors is a difficult task and often takes a significant amount of time in the software development life cycle. In the context of embedded software, the probability of bugs is quite high. Due to requirements of low code size and less resource consumption, embedded softwares typically do away with a lot of sanity checks during development time. This leads to high chance of errors being uncovered in the production code at run time. In this paper we propose a methodology for debugging errors in BusyBox, a de-facto standard for Linux in embedded systems. Our methodology works on top of Valgrind, a popular memory error detector and Daikon, an invariant analyzer. We have experimented with two published errors in BusyBox and report our findings in this paper.


Enumerating Subgraph Instances Using Map-Reduce

Abstract: The theme of this paper is how to find all instances of a given "sample" graph in a larger "data graph," using a single round of map-reduce. For the simplest sample graph, the triangle, we improve upon the best known such algorithm. We then examine the general case, considering both the communication cost between mappers and reducers and the total computation cost at the reducers. To minimize communication cost, we exploit the techniques of (Afrati and Ullman, TKDE 2011)for computing multiway joins (evaluating conjunctive queries) in a single map-reduce round. Several methods are shown for translating sample graphs into a union of conjunctive queries with as few queries as possible. We also address the matter of optimizing computation cost. Many serial algorithms are shown to be "convertible," in the sense that it is possible to partition the data graph, explore each partition in a separate reducer, and have the total computation cost at the reducers be of the same order as the computation cost of the serial algorithm.


MLLS: Minimum Length Link Scheduling Under Physical Interference Model

Abstract: We study a fundamental problem called Minimum Length Link Scheduling (MLLS) which is crucial to the efficient operations of wireless networks. Given a set of communication links of arbitrary length spread and assume each link has one unit of traffic demand in wireless networks, the problem MLLS seeks a schedule for all links (to satisfy all demands) of minimum number of time-slots such that the links assigned to the same time-slot do not conflict with each other under the physical interference model. In this paper, we will explore this problem under three important transmission power control settings: linear power control, uniform power control and arbitrary power control. We design a suite of new and novel scheduling algorithms and conduct explicit complexity analysis to demonstrate their efficiency. Our algorithms can account for the presence of background noises in wireless networks. We also investigate the fractional case of the problem MLLS where each link has a fractional demand. We propose an efficient greedy algorithm of the approximation ratio at most $(K+1)^{2}\omega$.