delphi - Reading large binary files -

-

I am working on a project where I am logging some signals (1d arrays) into a binary file . I can store large amounts of data (usually several giga bytes). Now I would like to reload those files

I have two visual elements, one is called an observation and the second is just a plain Y / T chart overview users should give the idea that the whole file (large File), while the chart only shows a part of the file which is usually run, which is selected in the observation with the resettable rectangle / band.

Because the files in the memory can be very large load, therefore not optimal, so the main idea is to load into memory (maximum up to several MBs) only to load and display important data / visual data. If the user zooms on the chart, then the data needs to be reloaded with more data points from the file.

My question is how to attract the viewing component to show the whole contents of the file best (do not actually load all the samples from the file) Suppose that my files are from 10 GB Are big and I would like to give users the idea that what is in the file but I can attract the maximum. 16k samples on observation component?

Is there any way to store any extra data (such as indexing, small data chunks, images ...) during logging for loading and drawing overview components later? Currently I'm only collecting samples, but adding additional data will not be a problem. Do you have any experience with how you did it?

To get an idea of ​​what I'm doing:

Enter the image details here

You get multi-level dilution - just every Nth, n ^ Store 2th and so many samples (for example, 10, 100, 1000, 10000 ...). When the user changes the window size, select the appropriate level, which contains approximately 1,000 samples in this window, load and show these samples (1000 digits is just the appropriate number for the chart on the screen).

If your data has some characteristics, specific properties, then it is possible to implement the Polyline Simplification algorithm to set up large (level) data and preserve features.

Comments

Post a Comment

Popular posts from this blog

Pass DB Connection parameters to a Kettle a.k.a PDI table Input step dynamically from Excel -

-
Image
I have a requirement whenever I run my own kettle, dynamically took database connection parameters from an excel source Every run should go. I have an Excel column names: hostname, username, database, password. I want to pass these connection parameters to my table input stage dynamically whenever it runs the job. This is what I was trying to do. You can get it Connection parameters from the DB reading source (for example Excel or a CSV file in my example) < / Li> Store the parameters in the variable Using the variable in your connection's settings. Setting Variables Create another conversion for (you can not do it in the same conversion Uses): set variables element variable: The element that reads / writes your data creates a new connection and sets the connection parameter using $ {variable name_} Note that you will need to type in the field field in the field $ {password} Also keep in mind that this can be a security problem as the
Read more

multithreading - PhantomJS-Node in a for Loop -

-
I am trying to implement a screen-scraps application that opens a URL in which a parameter changes as it is for a loop: var data = [ '100', '101', '102', '103', '104']; (Var indexA = 0; indexA & lt; data.length; indexA ++) for {Phantom Generation ({'port': freeport}, function (pH) {ph.create page (function (page) {page.open ("http: //...++ data [index B] +" ... " , Status {console.log ("open site:" + data [index B], position); / * OLD * / indexB ++; page evaluation (function () {// page scrape and an object Return}, function {console.log (result.dataRequest); // here is the problem}); / * new and fixed * / indexB ++} first console Logs it: Open Site: 100 Success Opens Site: 101 Success B Ola Site: 102 Success Opening Site: 103 Success Opened Site: 104 Success Which is good. Data is verified in scrap HTML and from index to array Matches. So in page 101 I scrap '101 and retrieve' and pu
Read more

c++ - MATLAB .m file to .mex file using Matlab Compiler -

-
I know that it can speak as a very famous topic, but something that helps me with it Not found This scenario is: I have my functions function, no matter what's inside, it's a generic function that may depend on other toolboxes. Does. Has input and output; I want to generate the same myFun.mex function; : Lib myFun This way I get some new files: libmyfun.c libmyfun.dll Libmyfun.exp libmyfun.exports libmyfun .h libmyfun.lib Reading around and I think that Creating a myfun.cpp that is very famous is included: / * gateway function * / zero max function (ent NLHS, MXARRA * PLS [], int NRHS, CONST M Xeray * PHS []) Here is the Variable Deals * / / * code here * /} I have tried to do this and are called Regular of Max Generation: Max ('- v', 'myfun.cpp') But I get many errors, and I should say that my myfun.cpp copy / Paste Function is where I have a lot of doubts ... especially on input / output management. My q
Read more