【问题标题】:Bash get value of matching substring in a long stringBash获取长字符串中匹配子字符串的值
【发布时间】:2020-07-16 20:44:38
【问题描述】:

我正在编写一个 bash 脚本,并且很难从匹配的子字符串中提取值。 我无法共享 curl 命令,因为它包含敏感信息,但会更新我遇到问题的 i 值(1)(更新为模拟真实值)。 jq 给出解析错误

curl -s -g "$line" | jq -c '.allBuilds[]' | while read i; do
   job_name=$(echo "$i" | jq .fullDisplayName | tr -d '»' | tr -s " " | sed 's/ /,/g' | tr -d '"')
done

预期输出 Hello,Java,World,master,#47

我得到了大多数 i 值的预期输出,但出现了一些错误

下面是样本 i 值。

1

{"_class":"org.jenkinsci.plugins.workflow.job.WorkflowRun","actions":[{"_class":"hudson.model.CauseAction"},{},{"_class":"hudson.model.ParametersAction","parameters":[{"_class":"hudson.model.StringParameterValue","name":"environment","value":"DE"},{"_class":"hudson.model.BooleanParameterValue","name":"update","value":false},{"_class":"hudson.model.BooleanParameterValue","name":"black","value":false},{"_class":"hudson.model.StringParameterValue","name":"description","value":"DE"},{"_class":"hudson.model.StringParameterValue","name":"number","value":""},{"_class":"hudson.model.StringParameterValue","name":"config","value":"{ "E": "DE", "Exp": "1111", "Pr": "D", "Man": { "Se": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/Se-B8SKMz", "OR": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/OR-kvJ2lJ", "AR": "arn:aws:secretsmanager:region:111111111111:secret:A/DE/SA/rds", "User": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/mUixbWY", "sales": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/saELY", "vau": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/vNRR7BO", "sc": "", "exAd": "arn:aws:secretsmanager:region:111111111111:secret:BA/A/ExyBoYL", "exp": "arn:aws:secretsmanager:region:111111111111:secret:BA/A/Exl67GE", "sec": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/Secle06a", "Secu": "arn:aws:secretsmanager:region:111111111111:secret:DE/A/Seia" }, "s3": { "buckets": { "hello": { "name": "helloDEBA", "region": "region", "account": "111111111111" }, "mlt": { "name": "sacdhbd", "region": "region", "account": "111111111111" }, "devo": { "name":"devvvvv", "region": "region", "account": "5555555" } } }, "roles": { "lam": "arn:aws:iam::111111111111:role/lam", "lambd": "arn:aws:iam::111111111111:role/lambd", "la": "arn:aws:iam::111111111111:role/lam", "la": "arn:aws:iam::111111111111:role/la", "la": "arn:aws:iam::111111111111:role/la","lasds": "arn:aws:iam::111111111111:role/lafgg", "lafdg": "arn:aws:iam::111111111111:role/dfsdv", "acc": "arn:aws:iam::111111111111:role/acc" }, "vpc": { "subnets": { "private": { "1a": "subnet-111111", "1b": "subnet-22222", "1c": "subnet-33333" } }, "securityGroupIds": { "lambda": "sg-1111" }, "endpoints": { "e": "" }, "links": { "b": "" } }, "securi": { "level": "FAILURE", "s": true }, "log": "debug", "se": "hello.com", "sa": { "env": "--DE" }, "lam": { "sss": { "environment": { "variables": { "test": "hello.com", "PhoneNumber": "11111" } } } } }"},{"_class":"hudson.model.BooleanParameterValue","name":"scan","value":false},{"_class":"hudson.model.BooleanParameterValue","name":"ch","value":false}]},{"_class":"jenkins.scm.A.SCMRevisionAction"},{},{"_class":"hudson.plugins.git.util.BuildData"},{"_class":"hudson.plugins.git.GitTagAction"},{},{},{},{"_class":"org.jenkinsci.plugins.workflow.cps.EnvActionImpl"},{"_class":"hudson.plugins.git.util.BuildData"},{},{},{},{},{},{"_class":"org.jenkinsci.plugins.pipeline.modeldefinition.actions.RestartDeclarativePipelineAction"},{},{"_class":"org.jenkinsci.plugins.workflow.job.views.FlowGraphAction"},{},{},{},{}],"fullDisplayName":"Hello » Java » World » master #25","id":"25","number":25,"timestamp":1575582153372}

2

{"_class":"org.jenkinsci.plugins.workflow.job.WorkflowRun","actions":[{"_class":"hudson.model.CauseAction"},{"_class":"hudson.model.ParametersAction","parameters":[{"_class":"hudson.model.BooleanParameterValue","name":"helo-world","value":false},{"_class":"hudson.model.StringParameterValue","name":"environment","value":"hello"},{"_class":"hudson.model.StringParameterValue","name":"config","value":""},{"_class":"hudson.model.StringParameterValue","name":"description","value":""},{"_class":"hudson.model.BooleanParameterValue","name":"hello","value":false},{"_class":"hudson.model.BooleanParameterValue","name":"hello2","value":false},{"_class":"hudson.model.BooleanParameterValue","name":"scan","value":false},{"_class":"hudson.model.StringParameterValue","name":"hello3","value":""}]},{"_class":"jenkins.scm.api.SCMRevisionAction"},{},{"_class":"hudson.plugins.git.util.BuildData"},{"_class":"hudson.plugins.git.GitTagAction"},{},{},{},{"_class":"org.jenkinsci.plugins.workflow.cps.EnvActionImpl"},{"_class":"hudson.plugins.git.util.BuildData"},{},{},{},{},{},{"_class":"org.jenkinsci.plugins.pipeline.modeldefinition.actions.RestartDeclarativePipelineAction"},{},{"_class":"org.jenkinsci.plugins.workflow.job.views.FlowGraphAction"},{},{},{},{}],"fullDisplayName":"Hello»Java»World»master#21","id":"21","number":21,"timestamp":1574705384077}

我正在尝试提取 fullDisplayName 值,并且尝试了一些选项,例如

printf '%s\n' "$i" | grep fullDisplayName

printf '%s\n' 在 shell 中尝试时以新行输出输出,但在脚本中其行为不同

{}
{}
{}]
fullDisplayName:Hello » Java » World » master #25
id:25
number:25
timestamp:1575582153372

【问题讨论】:

  • 你为什么要 grep 为 fullDisplayName 并期望得到输出 "Hello»Java»World»master#22","id":"22","number":22,"timestamp":1574705541773 显然不包含你 grep 的字符串 fullDisplayName
  • 你能发布curl命令的输出吗?
  • 每个"" 对中的"s 在curl -s -g ""$line"" 中的$line 周围相互抵消,所以什么都不做。 ""$line""$line 完全相同,因为它两侧的 "" 只是一个空字符串。我认为/希望你的意思是 "$line" 在两边都有单独的双引号(就像你在脚本后面对 "$i" 所做的那样)所以你引用 $line 而不是在两边多余地放置一个空字符串它。
  • @EdMorton 编辑了我的帖子(一直在尝试不同的选项)。
  • 您不能将格式化的文本放在评论中。请edit您的问题包括所有相关信息并做what @anubhava requested以便我们可以帮助您。

标签: bash jenkins awk sed jq


【解决方案1】:
  1. 看来您应该使用read -r

  2. 如果您专注于使用 curl 和 jq 来提取信息,而无需任何 grep 或 tr 调用并且没有任何 shell 循环,这似乎也会简单得多。假设您可以将 curl 的输出安排为有效的 JSON(*),按照以下几行对 jq 的单次调用应该可以完成这项工作:

jq -c '.allBuilds[] | .fullDisplayName | gsub("»";"")'

(*) 要检查curl 的输出是否有效,您可以将curl 命令的输出通过管道传送到jq empty

curl ... | jq empty

【讨论】:

  • 谢谢,在 curl 中阅读 -r 就可以了。没有 r,.allBuilds[] 不是有效的 json
猜你喜欢
  • 1970-01-01
  • 2015-10-14
  • 2014-11-18
  • 1970-01-01
  • 1970-01-01
  • 2013-08-12
  • 2018-08-19
  • 1970-01-01
  • 2021-08-25
相关资源
最近更新 更多