如何在没有内存耗尽的情况下读取大的 CSV 文件？

Question

我有一个超过 16 MB 的 CSV 文件。

当我读到它时：

$exportString = @file_get_contents($url, false, stream_context_create($contextOptions)

我只是想回应一下：

$data=explode(';', $exportString);
echo $data[0];

然后这条消息出现在我的浏览器中：

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 71 bytes) in....

然后，我想将它导入 MySQL DB。

有什么帮助吗？

Answer 1

在您的 php.ini 文件中，您可以增加允许的内存大小

memory_limit = 512M

或者在脚本的顶部放：

ini_set("memory_limit","512M");

或者如果你无权访问php.ini，在根目录下创建一个.htaccess文件并放入

php_value memory_limit = "512M"

---

编辑：268435456 字节 = 256MB，所以让它变大！

请记住，拥有巨大的内存限制并不能真正替代编写好的代码。最好使用file_get_contents 附加参数offset 和length 将其拆分为块。

---

拆分并非易事

但是，这里有一个简单的算法来告诉你如何做到这一点！

1. Initialize an empty string
  (begin a loop)
2. Grab a chunk from your file and append that to the string
3. Search for the last \n character in that string (MAKE SURE IT ISN'T PART OF DATA)
  a. If \n doesn't exist, continue
  b. If it does, grab the first substring up to that point and process that.
     Once finished grab the rest of substring assign it to your initial string.
  (loop until finished)
4. If there is data in the string left, do processing on that as well.

现在，寻找字符串中最后一个“\n”的算法

1. Initialize a variable called $inString = false and 
2. Initialize a variable $newLinePos = -1
3. Loop through each character of the string
  (begin loop)
  a. If the current charater is a double quote (")
     AND the character before IS NOT a backslash (\)
     Then set $inString = !$inString;
  b. If $inString Then continue;
  c. If the current character is the newline (\n)
     Then set $newLinePos to the index of the current character
  (end loop)
4. If $newLinePos == -1 then we have not found any valid \n and we need to grab more
   Otherwise, go on with the next part as perscribed above