arrays - PHP Greatly speed up execution time of my script -

-

I have a heavy script that we go a lot, the algorithm below has been used:

  1. Load 4500 rows from the database and store them as an array (A)

  2. Load 600000 rows from the database and place them in the array Store in the form. (B)

  3. (A) looks for a match (B) for each element.

  4. Next, the maximum amount of this script is 4500 * 60 000, which is 270 million, so you can understand that it may be a bit sweat for PHP.

    Can I make this process efficient in any way? Reading rows from the database is not really an issue, this array is iterative which brings huge cost.

    It works very fast, but a factor (60000) will increase greatly in the coming years.

    Any ideas?

    Here are a few different answers. My guess is that the first one is correct, easy and sufficient, But it is very difficult to make sure.

    Possible answer 1: Use of SQL

    As the comments indicate, it looks like a lot to be included. Apart from this, your post indicates that when a match is received, you only take one action and there are no ments matches in A. This means that your SQL statement should only return lines of matching, not all of them.

    Potential Answer 2: Sort Array

    Maybe you can do this, it does not matter that you can not do everything in SQL. Sort arrays (again, to your database to do this) Probably you can sort B so that the search for a match is sharp. Or enter the search value in the key of the array so that the search is very quick. Or if you are lucky then you may be able to sort both arrays in a manner that makes all A and B in the same order i.e. for any A you know that the right b is not in existence either Or B is not present later in the head.

    Possible answer 3: Tell more about the problem

    You have only given us your current algorithm, and not exactly what you are trying to do most likely it That's not the best idea to walk on everything, but no one can say so until they know more about your data and what you want to do at the end.

Comments

Post a Comment

Popular posts from this blog

Pass DB Connection parameters to a Kettle a.k.a PDI table Input step dynamically from Excel -

-
Image
I have a requirement whenever I run my own kettle, dynamically took database connection parameters from an excel source Every run should go. I have an Excel column names: hostname, username, database, password. I want to pass these connection parameters to my table input stage dynamically whenever it runs the job. This is what I was trying to do. You can get it Connection parameters from the DB reading source (for example Excel or a CSV file in my example) < / Li> Store the parameters in the variable Using the variable in your connection's settings. Setting Variables Create another conversion for (you can not do it in the same conversion Uses): set variables element variable: The element that reads / writes your data creates a new connection and sets the connection parameter using $ {variable name_} Note that you will need to type in the field field in the field $ {password} Also keep in mind that this can be a security problem as the
Read more

multithreading - PhantomJS-Node in a for Loop -

-
I am trying to implement a screen-scraps application that opens a URL in which a parameter changes as it is for a loop: var data = [ '100', '101', '102', '103', '104']; (Var indexA = 0; indexA & lt; data.length; indexA ++) for {Phantom Generation ({'port': freeport}, function (pH) {ph.create page (function (page) {page.open ("http: //...++ data [index B] +" ... " , Status {console.log ("open site:" + data [index B], position); / * OLD * / indexB ++; page evaluation (function () {// page scrape and an object Return}, function {console.log (result.dataRequest); // here is the problem}); / * new and fixed * / indexB ++} first console Logs it: Open Site: 100 Success Opens Site: 101 Success B Ola Site: 102 Success Opening Site: 103 Success Opened Site: 104 Success Which is good. Data is verified in scrap HTML and from index to array Matches. So in page 101 I scrap '101 and retrieve' and pu
Read more

c++ - MATLAB .m file to .mex file using Matlab Compiler -

-
I know that it can speak as a very famous topic, but something that helps me with it Not found This scenario is: I have my functions function, no matter what's inside, it's a generic function that may depend on other toolboxes. Does. Has input and output; I want to generate the same myFun.mex function; : Lib myFun This way I get some new files: libmyfun.c libmyfun.dll Libmyfun.exp libmyfun.exports libmyfun .h libmyfun.lib Reading around and I think that Creating a myfun.cpp that is very famous is included: / * gateway function * / zero max function (ent NLHS, MXARRA * PLS [], int NRHS, CONST M Xeray * PHS []) Here is the Variable Deals * / / * code here * /} I have tried to do this and are called Regular of Max Generation: Max ('- v', 'myfun.cpp') But I get many errors, and I should say that my myfun.cpp copy / Paste Function is where I have a lot of doubts ... especially on input / output management. My q
Read more