Ograniczanie wyników
Czasopisma help
Autorzy help
Lata help
Preferencje help
Widoczny [Schowaj] Abstrakt
Liczba wyników

Znaleziono wyników: 37

Liczba wyników na stronie
first rewind previous Strona / 2 next fast forward last
Wyniki wyszukiwania
Wyszukiwano:
w słowach kluczowych:  data compression
help Sortuj według:

help Ogranicz wyniki do:
first rewind previous Strona / 2 next fast forward last
EN
Data compression combined with effective encryption is a common requirement of data storage and transmission. Low cost of these operations is often a high priority in order to increase transmission speed and reduce power usage. This requirement is crucial for battery-powered devices with limited resources, such as autonomous remote sensors or implants. Well-known and popular encryption techniques are frequently too expensive. This problem is on the increase as machine-to-machine communication and the Internet of Things are becoming a reality. Therefore, there is growing demand for finding trade-offs between security, cost and performance in lightweight cryptography. This article discusses asymmetric numeral systems-an innovative approach to entropy coding which can be used for compression with encryption. It provides a compression ratio comparable with arithmetic coding at a similar speed as Huffman coding; hence, this coding is starting to replace them in new compressors. Additionally, by perturbing its coding tables, the asymmetric numeral system makes it possible to simultaneously encrypt the encoded message at nearly no additional cost. The article introduces this approach and analyzes its security level. The basic application is reducing the number of rounds of some cipher used on ANS-compressed data, or completely removing an additional encryption layer when reaching a satisfactory protection level.
PL
Metody hybrydowej kompresji wizji zrewolucjonizowały sposób zapisu oraz transmisji cyfrowego obrazu. Coraz wydajniejsza reprezentacja obrazu torowała drogę do szerokiego wdrożenia multimedialnych systemów kolejnych generacji, jak cyfrowa telewizja, kino domowe czy telewizja w Internecie. Jednak możliwości te nie pojawiły się z dnia na dzień. Są one wynikiem wielu lat badań, ulepszeń już istniejących rozwiązań, ale również wdrażania nowych pomysłów. W artykule przedstawiono skalę tego postępu w ciągu ostatnich dwóch dekad. Poddano ocenie kolejne generacje koderów obrazu, od MPEG-2 do nowej techniki HEVC, zarówno pod względem samych osiągów kompresji, jak również innych cech techniki, których nie sposób pominąć realizując jej wdrożenie.
EN
Hybrid video compression has revolutionized the way of a video saving and transmission. More and more efficient representation of a video paved the way for the broad implementation of multimedia systems of successive generations, such as digital television, home cinema or television over the Internet. However, these opportunities did not appear from day to day. They are the result of many years of research, improvements of existing solutions, but also the implementation of new ideas. The paper presents the scale of this progress over the last 2 decades. It evaluates the successive generations of video encoders, from MPEG-2 to the new HEVC technology, both in terms of its compression efficiency, as well as other technical features that can not be omitted while putting the technology into the market.
3
Content available remote Efficient Approaches to Compute Longest Previous Non-overlapping Factor Array
EN
In this article, we introduce new methods to compute the Longest Previous nonoverlapping Factor (LPnF) table. The LPnF table is the table that stores the maximal length of factors re-occurring at each position of a string without overlapping and this table is related to Ziv-Lempel factorization of a text which is useful for text compression and data compression. The LPnF table has the important role for data compression, string algorithms and computational biology. In this paper, we present three approaches to produce the LPnF table of a string from its augmented position heap, from its position heap, and from its suffix heap. We also present the experimental results from these three solutions. The algorithms run in linear time with linear memory space.
EN
Abstract:With growing demand for energy, power generated in renewable sources at various locations are distributed throughout the power grid. The power grid known as the smart grid needs to monitor power generation and its smart distribution. Smart meters provide solutions for monitoring power over smart grids. Smart meters need to continuously log data and at every source there is a large amount of data generated that needs to be compressed for both storage and transmission over the smart grid. In this paper, a novel algorithm for PQ data compression is proposed that uses the Dual Tree Complex Wavelet Transform (DTCWT) for sub-band computation and a modified quantizer is designed to reduce subband coefficient limits to less than 4 bits. The Run Length Encoding (RLC) and Huffman Coding algorithm encode the data further to achieve compression. The performance metrics such as a peak-signal-to-noise ratio (PSNR) and compression ratio (CR) are used for evaluation and it is found that the modified DTCWT (MDTCWT) improves PSNR by a factor of 3% and the mean squared error (MSE) by a factor of 16% as compared with the DTCWT based PQ compression algorithm.
EN
Image compression is an essential stage of the data archiving and transmitting process, as it reduces the number of bits and the time required to complete the transmission. In this article, a study of image transmission over the Multi-Carrier Code Division Multiple Access (MC-CDMA) downlink system is presented. The solution proposed relies on source coding combined with channel coding. The Discrete Wavelet Transform (DWT) method is used in conjunction with the SPIHT coder to compress the image, then the data generated is transmitted with the MC-CDMA technique over a noisy channel. The results show that image transmissions performed over MC-CDMA using the SPIHT model are better than the traditional approach like MC-CDMA in the AWGN channel.
EN
Analysis of patterns in binary matrices plays a vital role in numerous applications of computer science. One of the most essential patterns of such matrices are the so called switching components, where the number and location of the components gives valuable information about the binary matrix. One way to measure the effect of switching components in a binary matrix is counting the number of 0-s which have to be replaced with 1-s in order to eliminate the switching components. However, finding the minimal number of 0-1 flips is generally an NP-complete problem. We present two novel-type heuristics for the above problem and show via experiments that they outperform the formerly proposed ones, both in optimality and in running time. We also show how to use those heuristics for determining the so-called nestedness level of a matrix, and how to use the flips for binary image compression.
EN
Data gathered through seabed surveys performed using multibeam echosounder tend to be significant in size. Quite often a single measurement session leads to obtaining even several million distinct points (usually in x, y, z format). These data are saved in files (often text files), where x, y represent the location of a point (in geographical format, or more commonly in UTM format) and z represents the measured depth at the respective point. Due to the huge amount of such points, the data occupy a significant space in memory or in storage system (the order of megabytes for small areas and of gigabytes for larger ones). The paper contains a survey of existing methods of compressing ASCII UTM files and a proposal of a novel method tailored for a particular data structure. As a result of utilising differential coding and coding using varying length values, the size of such files can be diminished by a factor exceeding ten, while preserving the full information. The paper presents a detailed description of the proposed algorithm and experimental results using real data.
PL
Dane pozyskane z sondaży dna morskiego wykonane z użyciem sondy wielowiązkowej cechują się znacznym rozmiarem. Bardzo często w wyniku jednej sesji pomiarowej otrzymujemy nawet kilka milionów pojedynczych punktów (najczęściej w formacie x,y,z). Informacje te zapisywane są w plikach, często tekstowych, gdzie x,y to położenie punktu (w formacie geograficznym lub częściej UTM), a z określa zmierzoną głębokość w tym punkcie. Ze względu na ogromną liczbę tych punktów dane te zajmują w pamięci komputera lub na dyskach znaczny rozmiar (liczony w MB dla małych obszarów lub GB dla większych). Autorzy przedstawili w artykule różne metody kompresji plików ASCII UTM, w tym opracowaną autorską metodę dopasowaną do struktury danych. Dzięki zastosowaniu metody zapisu różnicowego z wykorzystaniem zmiennej długości w bajtach możemy ponad dziesięciokrotnie zmniejszyć rozmiary tego typu plików, przy zachowaniu pełnej informacji. W artykule przedstawiono dokładny algorytm oraz testy wykonane na danych rzeczywistych.
8
Content available remote 2D DCT compression in the switched-current technique
EN
The article presents a methodology for designing an analogue processor for a DCT compression using methods and strategies for designing digital circuits: the row strategy, a standard digital router and an automatic synthesis of architecture from its description in a VHDL-AMS language. The correctness of work of the topography has been verified with post-layout simulations of processing an exemplary image in the compressing task, using the discrete cosine transform. The quality of processing has been compared with other solutions available in literature by calculating the PSNR and Accuracy coefficients for the processed image. The article also presents changes of the PSNR coefficient depending on the level of the applied compression.
PL
W artykule zaprezentowana została metodologia projektowania analogowego procesora kompresji DCT z wykorzystaniem metod i strategii projektowania układów cyfrowych: strategii wierszowej, standardowego cyfrowego routera oraz metod automatycznej syntezy architektury z jej opisu w języku VHDL-AMS. Poprawność działania topografii zweryfikowana została symulacjami post-layoutowymi procesu przetwarzania przykładowego obrazu w zadaniu jego kompresji za pomocą dyskretnej transformaty kosinusowej. Jakość przetwarzania porównana została z innymi rozwiązaniami dostępnymi w literaturze poprzez wyliczenie współczynników PSNR oraz Accuracy dla przetworzonego obrazu. W artykule zaprezentowano również zmiany współczynnika PSNR w zależności od stopnia zastosowanej kompresji.
PL
Prezentowane w pracy badania dotyczą bezstratnej kompresji danych opartej o metodę Huffmana i zgodnej ze standardem deflate stosowanym w plikach .zip / .gz. Zaproponowana jest optymalizacja kodera Huffmana polegająca na podziale na bloki, w których stosuje się różne książki kodowe. Wprowadzenie dodatkowego bloku z reguły poprawia stopień kompresji kosztem narzutu spowodowanego koniecznością przesłania dodatkowej książki kodowej. Dlatego w artykule zaproponowano nowy algorytm podziału na bloki.
EN
According to deflate [2] standard (used e.g. in .zip / .gz files), an input file can be divided into different blocks, which are compressed employing different Huffman [1] codewords. Usually the smaller the block size, the better the compression ratio. Nevertheless each block requires additional header (codewords) overhead. Consequently, introduction of a new block is a compromise between pure data compression ratio and headers size. This paper introduces a novel algorithm for block Huffman compression, which compares sub-block data statistics (histograms) based on current sub-block entropy E(x) (1) and entropy-based estimated average word bitlength Emod(x) for which codewords are obtained for the previous sub-block (2). When Emod(x) - E(x) > T (T - a threshold), then a new block is inserted. Otherwise, the current sub-block is merged into the previous block. The typical header size is 50 B, therefore theoretical threshold T for different sub-block sizes S is as in (3) and is given in Tab. 2. Nevertheless, the results presented in Tab. 1 indicate that optimal T should be slightly different - smaller for small sub-block size S and larger for big S. The deflate standard was selected due to its optimal compression size to compression speed ratio [3]. This standard was selected for hardware implementation in FPGA [4, 5, 6, 7].
PL
W artykule przedstawiono metody ograniczenia ilości danych podczas zautomatyzowanego pomiaru charakterystyk częstotliwościowych układów elektronicznych poprzez optymalizację rozdzielczości w dziedzinie częstotliwości oraz grupowanie i uśrednianie wyników w ramach ustalonych przedziałów. Efektem przedstawionej metodyki jest redukcja danych pomiarowych i udokładnienie charakterystyk w obszarach o podwyższonej niepewności.
EN
In this paper, there are presented two methods limiting the amount of data during an automated measurement of the frequency response characteristics of linear electric circuits. The first method involves the optimization of the resolution in the frequency domain. It consists in the usage of a changeable frequency of measurements dependent on the gradient of the characteristics of the measured circuit. The frequency of measurements is automatically regulated so that the absolute difference between the values of the subsequent measurements is approximately constant. The second method involves the reduction of data in the areas with the increased measurement uncertainty, with the standard method of increasing the number of measurements. The method requires division of the frequency range with the increased measurement number into intervals, grouping and averaging the data in these intervals. The aforementioned techniques can be applied in parallel, integrating them into a single system. The result of the described integrated methodology is the decrease in the number of measurement data files and frequently decrease in the overall experiment time without significant decrease in the quality of the frequency characteristics reconstruction. Depending on the assumed quality and characteristic of the measurement, the amount of data can be reduced two to ten times. Moreover, the accuracy of the characteristic areas with the increased measurement uncertainty can be increased with the averaging method without the increase in the number of the data. The presented methodology can be implemented in computer measurement systems.
EN
Systems based on principal component analysis have developed from exploratory data analysis in the past to current data processing applications which encode and decode vectors of data using a changing projection space (eigenspace). Linear systems, which need to be solved to obtain a constantly updated eigenspace, have increased significantly in their dimensions during this evolution. The basic scheme used for updating the eigenspace, however, has remained basically the same: (re)computing the eigenspace whenever the error exceeds a predefined threshold. In this paper we propose a computationally efficient eigenspace updating scheme, which specifically supports high-dimensional systems from any domain. The key principle is a prior selection of the vectors used to update the eigenspace in combination with an optimized eigenspace computation. The presented theoretical analysis proves the superior reconstruction capability of the introduced scheme, and further provides an estimate of the achievable compression ratios.
PL
Niniejszy artykuł opisuje nową architekturę sprzętową kompresji słownikowej, np. LZ77, LZSS czy też Deflate. Zaproponowana architektura oparta jest na funkcji haszującej. Poprzednie publikacje były oparte na sekwencyjnym odczycie adresu wskazywanego przez pamięć hasz, niniejszy artykuł opisuje układ, w którym możliwe jest równoległe odczytywanie tego adresu z wielu pamięci hasz, w konsekwencji możliwa jest kompresja słownikowa z szybkością na poziomie 1B ciągu wejściowego na takt zegara. Duża szybkość kompresji jest okupiona nieznacznym spadkiem stopnia kompresji.
EN
This paper describes a novel parallel architecture for hardware (ASIC or FPGA) implementation of dictionary compressor, e.g. LZ77 [1], LZSS [2] or Deflate [4]. The proposed architecture allows for very fast compression – 1B of input data per clock cycle. A standard compression architecture [8, 9] is based on sequential hash address reading (see Fig. 2) and requires M clock cycles per 1B of input data, where M is the number of candidates for string matching, i.e. hashes look ups (M varies for different input data). In this paper every hash address is looked up in parallel (see Fig. 3). The drawback of the presented method is that the number of M is defined (limited), therefore the compression ratio is slightly degraded (see Fig. 4). To improve compression ratio, a different sting length may be searched independently, i.e. not only 3B, but also 4B, … N B hashes (see results in Fig. 5, 6). Every hash memory (M(N-2)) usually requires a direct look-up in the dictionary to eliminate hash false positive cases or to check whether a larger length sting was found. In order to reduce the number of dictionary reads, an additional pre-elimination algorithm is proposed, thus the number of dictionary reads does not increase rapidly with growing N (see Fig. 7).
PL
W artykule omówiono zastosowanie analizy składników głównych (PCA) w zadaniu kompresji stratnej sygnału na przykładzie kompresji obrazu. Zadanie zrealizowano z wykorzystaniem klasycznej metody PCA oraz dwóch rodzajów sieci neuronowych: jednokierunkowej, dwuwarstwowej sieci z uczeniem nadzorowanym i jednowarstwowej sieci z uczeniem nienadzorowanym. W każdym z przypadków przeanalizowano wpływ struktury modelu PCA na wartości współczynnika kompresji oraz średniokwadratowego błędu kompresji.
EN
In the paper, lossy data compression techniques based on the principal component analysis (PCA) are considered on the example of image compression. The presented task is performed using the classical PCA method based on the eigen-decomposition of the image covari-ance matrix as well as two different kinds of artificial neural networks. The first neural structure used is a two-layer feed-forward network with supervised learning shown in Fig.1, while the second one is a single-layered network with unsupervised Hebbian learning. In each case considered, the effect of the PCA model structure on the data compression ratio and the mean square reconstruction error is analysed. The compression results for a Hebbian neural network with K=4 PCA units are presented in Figs. 2, 3 and 4. They show that only 4 eigenvectors are able to capture the main features of the processed image, giving as a result high value of the data compression ratio. However, the reconstructed image quality is not sufficient from a practical point of view. Therefore, selection of the appropriate value for K should take into account the tradeoff between a sufficiently high value for the compression ratio and a reasonably low value for the image reconstruction error. The summary results for both classical and neural PCA compression approaches obtained for different number of eigenvectors (neurons) are compared in Fig. 5. The author concludes that a positive aspect of using neural networks as a tool for extracting principal components from the image data is that they do not require calculating the correlation matrix explicitly, as in the case of the classical PCA-based approach.
PL
Otwarty standard kompresji danych, Deflate, jest szeroko stosowanym standardem w plikach .gz / .zip i stanowi kombinację kompresji metodą LZ77 / LZSS oraz kodowania Huffmana. Niniejszy artykuł opisuje implementację w układach FPGA dekompresji danych według tego standardu. Niniejszy moduł jest w stanie dokonać dekompresji co najmniej 1B na takt zegara, co przy zegarze 100MHz daje 100MB/s. Aby zwiększyć szybkość, możliwa jest praca wielu równoległych modułów dla różnych strumieni danych wejściowych.
EN
This paper describes FPGA implementation of the Deflate standard decoder. Deflate [1] is a commonly used compression standard employed e.g. in zip and gz files. It is based on dictionary compression (LZ77 / LZSS) [4] and Huffman coding [5]. The proposed Huffman decoded is similar to [9], nevertheless several improvements are proposed. Instead of employing barrel shifter a different translation function is proposed (see Tab. 1). This is a very important modification as the barrel shifter is a part of the time-critical feedback loop (see Fig. 1). Besides, the Deflate standard specifies extra bits, which causes that a single input word might be up to 15+13=28 bits wide, but this width is very rare. Consequently, as the input buffer might not feed the decoder width such wide input date, a conditional decoding is proposed, for which the validity of the input data is checked after decoding the input symbol, thus when the actual input symbol bit widths is known. The implementation results (Tab. 2) show that the occupied hardware resources are mostly defined by the number of BRAM modules, which are mostly required by the 32kB dictionary memory. For example, comparable logic (LUT / FF) resources to the Deflate standard decoder are required by the AXI DMA module which transfers data to / from the decoder.
PL
Praca opisuje zmodyfikowany sposób budowania książki kodowej kodu Huffmana. Książka kodowa została zoptymalizowana pod kątem implementacji sprzętowej kodera i dekodera Huffmana w układach programowalnych FPGA. Opisano dynamiczną metodę kodowania - książka kodowa może się zmieniać w zależności od zmiennego formatu kompresowanych danych, ponadto musi być przesłana z kodera do dekodera. Sprzętowa implementacja kodeka Huffmana wymusza ograniczenie maksymalnej długości słowa, w przyjętym założeniu do 12 bitów, co pociąga za sobą konieczność modyfikacji algorytmu budowy drzewa Huffmana.
EN
This paper presents a modified algorithm for constructing Huffman codeword book. Huffman coder, decoder and histogram calculations are implemented in FPGA similarly like in [2, 3]. In order to reduce the hardware resources the maximum codeword is limited to 12 bit. It reduces insignificantly the compression ratio [2, 3]. The key problem solved in this paper is how to reduce the maximum codeword length while constructing the Huffman tree [1]. A standard solution is to use a prefix coding, like in the JPEG standard. In this paper alternative solutions are presented: modification of the histogram or modification of the Huffman tree. Modification of the histogram is based on incrementing (disrupting) the histogram values for an input codeword for which the codeword length is greater than 12 bit and then constructing the Huffman tree from the very beginning. Unfortunately, this algorithm is not deterministic, i.e. it is not known how much the histogram should be disrupted in order to obtain the maximum codeword length limited by 12 bit. Therefore several iterations might be required. Another solution is to modify the Huffman tree (see Fig. 2). This algorithm is more complicated (when designing), but its execution time is more deterministic. Implementation results (see Tab. 1) show that modifi-cation of the Huffman tree results in a slightly better compression ratio.
16
Content available remote Optimal physical primaries of spectral color information
EN
A new method for selection of optimal physical primaries is introduced. The reflectance spectra of 1269 matt Munsell color chips are used as a universal physical dataset and the most independent samples are extracted and used as actual primaries. The efficiency of selected primaries is compared with those obtained from principal component analysis (PCA), non-negative matrix factorization (NNMF) and non-linear principal component analysis (NLPCA) techniques. The performances of chosen primaries are evaluated by calculation of root mean square (RMS) errors, the goodness of fit coefficient (GFC) and the color difference (ΔE) values between the original and the reconstructed spectra.
17
Content available remote Google Books Ngrams Recompressed and Searchable
EN
One of the research fields significantly affected by the emergence of “big data” is computational linguistics. A prominent example of a large dataset targeting this domain is the collection of Google Books Ngrams, made freely available, for several languages, in July 2009. There are two problems with Google Books Ngrams the textual format (compressed with Deflate) in which they are distributed is highly inefficient we are not aware of any tool facilitating search over those data, apart from the Google viewer, which, as a Web tool, has seriously limited use. In this paper we present a simple preprocessing scheme for Google Books Ngrams, enabling also search for an arbitrary n gram (i.e., its associated statistics) in average time below 0.2 ms. The obtained compression ratio, with Deflate (zip) left as the backend coder, is over 3 times higher than in the original distribution.
19
Content available remote Neural networks for the analysis of mine-induced building vibrations
EN
A study of the capabilities of arti?cial neural networks in respect of selected problems of the analysis of mine-induced building vibrations is presented. Neural network technique was used for the prediction of building fundamental natural period, mapping of mining tremors parameters into response spectra from ground vibrations, soil-structure interaction analysis, simulation of building response to seismictype excitation. On the basis of the experimental data obtained from the measurements of kinematic excitations and dynamic responses of actual structures, training and testing patterns of neural networks were formulated. The obtained results lead to a conclusion that the neural technique gives possibility of e?cient, accurate enough for engineering, analysis of structural dynamics problems related to mineinduced excitations.
PL
W pracy przedstawiono projekt oraz implementację systemu przeznaczonego do kompresji danych z sonarów wielowiązkowych działającego z wykorzystaniem technologii CUDA. Omówiono oraz zastosowano metody bezstratnej kompresji danych oraz techniki przetwarzania równoległego. Stworzoną aplikację przetestowano pod kątem szybkości i stopnia kompresji oraz porównano z innymi rozwiązaniami umożliwiającymi kompresję tego typu informacji.
EN
Recently, multibeam echosounders capable of logging, not only bathymetry focused data, but also the full water-column information have become available. Unlike using bathymetric multibeam sonars, which only capture the seafloor, utilizing full water-column multibeam systems capabilites results in acquiring very large data sets during hydrographic or scientific cruises. The paper presents the concept of algorithms dedicated for reduction of multibeam sonar datasets based on aplying multi-threaded architecture implemented in Graphical Processing Units (GPU). We presented the advantages of utilizing nVdia CUDA technology in the context of efficiency of compression and obtained data reduction ratio.
first rewind previous Strona / 2 next fast forward last
JavaScript jest wyłączony w Twojej przeglądarce internetowej. Włącz go, a następnie odśwież stronę, aby móc w pełni z niej korzystać.