在 venv 中通过 bash 运行 python 脚本时出现意外失败

Question

我有这个 bash 脚本：

#!/bin/bash
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s BTC/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s ETH/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s XRP/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s LTC/USDT
sleep 1
/home/reallymemorable/Documents/environments/trading/bin/python3 1-ccxt_2.py -e bittrex -s BCH/USDT

sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 2-dateConvert.py
sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 3-concatCrypto.py
sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 4-cutDates_1year.py "$(date +%Y-%m-%d -d "1 year ago")" "$(date +'%Y-%m-%d')" < DailyCryptoPricesFull.csv
sleep 3
/home/reallymemorable/Documents/environments/trading/bin/python3 4-cutDates_3months.py "$(date +%Y-%m-%d -d "3 months ago")" "$(date +'%Y-%m-%d')" < DailyCryptoPricesFull.csv

sleep 1
mkdir $(date +"%Y-%m-%d")
sleep 1
mv -- *.csv "$(date +'%Y-%m-%d')"

它提取 5 个不同的加密时间序列，在 dateConvert.py 脚本中转换日期格式，在 concatCrypto.py 脚本中加入所有 5 个加密，然后生成 1 年前和 3 个月前的 2 个版本的输出。然后它会创建一个以当前日期为名称的目录，并将所有文件移入其中。

这一切都很完美，但后来我意识到我需要在一个 venv 中完成这一切。我创建了 venv 并更改了我的 python 调用（如上所示）以使用 venv。

但是现在发生的是第一组命令正确执行（5 个数据提取），但其余的 python 脚本根本没有执行。 （mkdir 和 mv 命令执行正确。）

我收到以下错误：

Traceback (most recent call last):
  File "3-concatCrypto.py", line 33, in <module>
    df_merged = reduce(lambda  left,right: pd.merge(left,right,on=['Timestamp'], how='outer'), data_frames).fillna('NULL')
TypeError: reduce() of empty sequence with no initial value
getCrypto.sh: 33: getCrypto.sh: cannot open DailyCryptoPricesFull.csv: No such file
getCrypto.sh: 37: getCrypto.sh: cannot open DailyCryptoPricesFull.csv: No such file

这表明它已经开始在 concatCrypto.py 文件上失败，但我知道它实际上在此之前失败（真的，跳过）脚本，因为日期没有转换。

我猜这与您在 venvs 中引用路径的方式有关，但我不确定。

这里是 dateConvert.py 的内容：

import csv
import datetime
import time
import pandas as pd
import glob

path = "/home/reallymemorable/Documents/scripts/Crypto/*1d.csv"
for fname in glob.glob(path):
    print(fname)
    # Step 1: Convert the dates
    df = pd.read_csv(fname)
    df['Timestamp'] = pd.to_datetime(df['Timestamp'], unit='ms')
    # print(df.head())
    # nameWithDate = str(str(datetime.date.today()) + fname)
    df.to_csv(fname + '-dateFormatted.csv', index=False)

还有 concatCrypto.py：

import csv
import datetime
import time
import pandas as pd
import glob
from functools import reduce
import os
from datetime import date


data_frames = []

for fname in glob.glob("/home/reallymemorable/Documents/scripts/Crypto/*dateFormatted.csv"):
   data_frames.append(pd.read_table(fname, sep=','))

df_merged = reduce(lambda  left,right: pd.merge(left,right,on=['Timestamp'], how='outer'), data_frames).fillna('NULL')
print(df_merged)
df_sorted = df_merged.sort_values(by='Timestamp')
print(df_sorted)
# df_oneYear = df_sorted[df_sorted['Timestamp'] == '2019-06-11']
# print(df_oneYear)
df_sorted.to_csv('DailyCryptoPricesFull.csv', index=False)

编辑：

错误似乎是glob没有在dateConvert.py中运行。

我尝试手动安装，但收到此错误：

reallymemorable@gort:~/Documents/environments$ python3 --version
Python 3.6.9
reallymemorable@gort:~/Documents/environments$ python --version
Python 2.7.17
reallymemorable@gort:~/Documents/environments$ source mai_trading/bin/activate
(mai_trading) reallymemorable@gort:~/Documents/environments$ python --version
Python 2.7.17
(mai_trading) reallymemorable@gort:~/Documents/environments$ python3 --version
Python 3.6.9
(mai_trading) reallymemorable@gort:~/Documents/environments$ pip3 install glob
Collecting glob
Exception:
Traceback (most recent call last):
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/home/reallymemorable/Documents/environments/trading/lib/python3.6/site-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/home/reallymemorable/Documents/environments/trading/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/glob/

Answer 1

编辑：facepalm我已经更改了目录的名称并硬编码了 glob 调用中的路径。