为什么 Marathon 调度程序在重启 marathon 服务时会重启部署？答案

【问题标题】：Why does Marathon scheduler restart deployments upon restarting marathon service?为什么 Marathon 调度程序在重启 marathon 服务时会重启部署？
【发布时间】：2018-09-26 16:59:51
【问题描述】：

我在马拉松调度程序中测试我的工作并观察到马拉松调度程序在重新启动马拉松服务（systemctl restart marathon.service）时重新启动我完成的工作。不确定我是否缺少任何会阻止此行为的配置。我希望作业运行一次并完成。

测试设置 - 我使用 marathon 作为调度程序在 mesos 集群上运行作业。使用带有 force=true 标志的 rest api 将作业配置发布到 marathon。

作业预计运行一次即可完成。

作业 json -

{
  "id": "/test-job",
  "cmd": "/bin/ls",
  "cpus": 0.25,
  "mem": 100,
  "disk": 100,
  "instances": 1,
  "acceptedResourceRoles": [
    "mesos-workers"
  ],
  "labels": {
    "MARATHON_SINGLE_INSTANCE_APP": "true",
  },
  "portDefinitions": [],
  "user": "nobody",
  "backoffSeconds": 2147483647,
  "maxLaunchDelaySeconds": 2147483647,
}

还通过在升级策略的作业定义中添加额外参数来测试重启马拉松 -

  "upgradeStrategy": {
    "maximumOverCapacity": 0,
    "minimumHealthCapacity": 0
  }

我们将不胜感激任何帮助确定可能出错的地方。

谢谢！

重启马拉松服务后的马拉松日志-

Sep 25 20:45:04 10.162.217.171 marathon[2801]: [2018-09-25 20:45:04,878] INFO  removing matcher ActorOfferMatcher(Actor[akka://marathon/user/launchQueue/1/0-test-job#203351593]) (mesosphere.marathon.core
Sep 25 20:45:04 10.162.217.171 marathon[2801]: [2018-09-25 20:45:04,891] INFO  Processing LaunchEphemeral(Instance(instance [test-job.marathon-e18878ba-c103-11e8-a594-12d685c81d52],AgentInfo(10.162.147.2
Sep 25 20:45:04 10.162.217.171 marathon[2801]: [2018-09-25 20:45:04,905] INFO  Finished processing 1bf99832-7f87-4609-b591-8261ed4739eb-O630667 from 10.162.147.203. Matched 1 ops after 2 passes. First 10: cpus(
Sep 25 20:45:04 10.162.217.171 marathon[2801]: [2018-09-25 20:45:04,948] WARN  The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. (org.apache.curato
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,113] INFO  Received status update for task test-job.e18878ba-c103-11e8-a594-12d685c81d52: TASK_STARTING () (mesosphere.marathon.Maratho
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,145] INFO  Acknowledge status update for task test-job.e18878ba-c103-11e8-a594-12d685c81d52: TASK_STARTING () (mesosphere.marathon.core
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,149] INFO  Received status update for task test-job.e18878ba-c103-11e8-a594-12d685c81d52: TASK_RUNNING () (mesosphere.marathon.Marathon
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,163] INFO  Acknowledge status update for task test-job.e18878ba-c103-11e8-a594-12d685c81d52: TASK_RUNNING () (mesosphere.marathon.core.
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,405] INFO  Received status update for task test-job.e18878ba-c103-11e8-a594-12d685c81d52: TASK_FINISHED (Command exited with status 0)
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,409] INFO  all tasks of instance [test-job.marathon-e18878ba-c103-11e8-a594-12d685c81d52] are terminal, requesting to expunge (mesosphe
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,426] INFO  Removed app [/test-job] from tracker (mesosphere.marathon.core.task.tracker.InstanceTracker$InstancesBySpec:marathon-akka.ac
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,430] INFO  receiveInstanceUpdate: instance [test-job.marathon-e18878ba-c103-11e8-a594-12d685c81d52] was deleted (Finished) (mesosphere.
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,432] INFO  initiating a scale check for runSpec [/test-job] due to [instance [test-job.marathon-e18878ba-c103-11e8-a594-12d685c8
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,433] INFO  Acknowledge status update for task test-job.e18878ba-c103-11e8-a594-12d685c81d52: TASK_FINISHED (Command exited with status
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,436] INFO  Increasing delay. Task launch delay for [/test-job - 2018-09-24T21:51:25.894Z] is set to 24855 days 3 hours 14 minutes 7 sec
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,437] INFO  Need to scale /test-job from 0 up to 1 instances (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0)
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,446] INFO  Stopped InstanceLauncherActor for /test-job version 2018-09-24T21:51:25.894Z (mesosphere.marathon.core.launchqueue.impl.Task
Sep 25 20:45:05 10.162.217.171 marathon[2801]: [2018-09-25 20:45:05,450] WARN  Got unexpected terminated for runSpec /test-job: Actor[akka://marathon/user/launchQueue/1/0-test-job#203351593] (meso

【问题讨论】：

标签： mesos marathon

【解决方案1】：

Marathon 应该是一个用于长时间运行的应用程序/进程的框架（因此得名 Marathon）。换句话说，它不适合计划的或一次性的作业/流程。为了过度简化，基本上 Marathon 在每个应用程序的无限循环中执行以下操作

IF    number of instances running != number of instance desired
THEN  launch/kill instances to make sure number of instances running == number of instance desired
ELSE  do nothing

所以不管你是否重启Marathon，如果前一个任务完成，它就会开始一个新的任务

我建议您改用旨在启动作业的框架之一：

计时码表：https://mesos.github.io/chronos/

厨师：https://github.com/twosigma/Cook

节拍器：https://github.com/dcos/metronome

【讨论】：