如何在 Scala 中使用 Heroku DATABASE_URL 环境变量？

Question

我正在尝试将我的应用程序部署到 Heroku。 Heroku 提供了一个 DATABASE_URL 环境变量。我想知道使用它的最佳方法是什么。

在 dev 中，DATABASE_URL 环境变量将为 jdbc:postgresql://localhost:5400/bookswapdb

在 prd 中，DATABASE_URL 环境变量将是这样的：postgres://<USER>:<PASSWORD>@<HOST>:<PORT>/<DATABASE>

1. 检查 DATABASE_URL 真的是发现应用程序是在 prod 还是 dev 中的最佳方法吗？ （我原以为他们也会传入像 ENV=prd 或 PRD=true 这样的变量）

2. 鉴于下面的代码，实现检查 DATABASE_URL 并将其传递给 initFlyway 函数的最聪明的方法是什么。

3. 切碎这个字符串以获得我需要的详细信息的最干净的方法是什么postgres://<USER>:<PASSWORD>@<HOST>:<PORT>/<DATABASE>

4. DATABASE_URL 可能采用这种格式：jdbc:postgresql://<host>:<port>/<dbname>?user=<username>&password=<password> 如果是这样，最好的划分方法是什么？

以下内容在部署到 heroku 时似乎不起作用，但在本地数据库上确实有效。

package com.fullstackryan.appone.server

import cats.effect.{ConcurrentEffect, ContextShift, Sync, Timer}
import cats.implicits._
import com.fullstackryan.appone.config.{Config, DbConfig, LoadConfig, ServerConfig}
import com.fullstackryan.appone.database.Database
import com.fullstackryan.appone.repo.{BookSwap, HelloWorld, Jokes}
import com.fullstackryan.appone.routing.ApponeRoutes
import fs2.Stream
import org.flywaydb.core.Flyway
import org.http4s.client.blaze.BlazeClientBuilder
import org.http4s.implicits._
import org.http4s.server.blaze.BlazeServerBuilder
import org.http4s.server.middleware.Logger
import pureconfig.generic.auto._

import java.net.URI
import scala.concurrent.ExecutionContext.global


object ApponeServer {

  def initFlyway[F[_] : Sync](url: String, username: String, password: String): F[Int] = Sync[F].delay {
    val flyway = Flyway.configure().dataSource(url, username, password).baselineOnMigrate(true).load()
    println("inside flyway")
    flyway.migrate()
  }

  def prodConfig(): Config = {
        val dbUri = new URI(System.getenv("DATABASE_URL"))
        val username = dbUri.getUserInfo.split(":")(0)
        val password = dbUri.getUserInfo.split(":")(1)
        val dbUrl = "jdbc:postgresql://" + dbUri.getHost + dbUri.getPath

        Config(ServerConfig(5432, dbUri.getHost), DbConfig(dbUrl, username, password, 10))
  }

  def stream[F[_] : ConcurrentEffect : ContextShift : Timer]: Stream[F, Nothing] = {
    for {
      client <- BlazeClientBuilder[F](global).stream
      // below line loads config from application.conf
      config <- Stream.eval(LoadConfig[F, Config].load)
      // This is meant to check if DATABASE_URL is dev or prd
      isProdConfig = if (config.dbConfig.url.contains("localhost")) config else prodConfig()
      // Below line hopefully passes correct prd or dev config into initFlyway to get a connnection 
      _ <- Stream.eval(initFlyway(isProdConfig.dbConfig.url, isProdConfig.dbConfig.username, isProdConfig.dbConfig.password))
      xa <- Stream.resource(Database.transactor(isProdConfig.dbConfig))
      helloWorldAlg = HelloWorld.impl[F]
      jokeAlg = Jokes.impl[F](client)
      bookAlg = BookSwap.buildInstance[F](xa)
      httpApp = (
        ApponeRoutes.helloWorldRoutes[F](helloWorldAlg) <+>
          ApponeRoutes.bookRoutes[F](bookAlg) <+>
          ApponeRoutes.jokeRoutes[F](jokeAlg)
        ).orNotFound

      finalHttpApp = Logger.httpApp(true, true)(httpApp)

      exitCode <- BlazeServerBuilder[F](global)
        .bindHttp(8080, "0.0.0.0")
        .withHttpApp(finalHttpApp)
        .serve
    } yield exitCode
  }.drain
}

错误

2021-01-17T14:05:29.437876+00:00 heroku[web.1]: State changed from crashed to starting
2021-01-17T14:05:35.808791+00:00 heroku[web.1]: Starting process with command `target/universal/stage/bin/appone -Dhttp.port=${PORT}`
2021-01-17T14:05:39.711887+00:00 app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
2021-01-17T14:05:40.047036+00:00 app[web.1]: Picked up JAVA_TOOL_OPTIONS: -Xmx300m -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8
2021-01-17T14:05:48.838694+00:00 app[web.1]: [ioapp-compute-0] INFO o.h.c.PoolManager - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false
2021-01-17T14:05:48.972995+00:00 app[web.1]: pureconfig.error.ConfigReaderException: Cannot convert configuration to a scala.runtime.Nothing$. Failures are:
2021-01-17T14:05:48.973019+00:00 app[web.1]: at 'appone.db-config':
2021-01-17T14:05:48.973021+00:00 app[web.1]: - (application.conf @ jar:file:/app/target/universal/stage/lib/com.fullstackryan.appone-0.0.1-SNAPSHOT.jar!/application.conf: 10) Key not found: 'username'.
2021-01-17T14:05:48.973022+00:00 app[web.1]: - (application.conf @ jar:file:/app/target/universal/stage/lib/com.fullstackryan.appone-0.0.1-SNAPSHOT.jar!/application.conf: 10) Key not found: 'password'.
2021-01-17T14:05:48.977706+00:00 app[web.1]:
2021-01-17T14:05:48.977988+00:00 app[web.1]: at com.fullstackryan.appone.config.LoadConfig$$anon$1.$anonfun$load$1(LoadConfig.scala:25)
2021-01-17T14:05:48.978182+00:00 app[web.1]: at cats.syntax.EitherOps$.leftMap$extension(either.scala:172)
2021-01-17T14:05:48.992062+00:00 app[web.1]: at com.fullstackryan.appone.config.LoadConfig$$anon$1.load(LoadConfig.scala:25)
2021-01-17T14:05:48.992224+00:00 app[web.1]: at com.fullstackryan.appone.server.ApponeServer$.$anonfun$stream$1(ApponeServer.scala:50)
2021-01-17T14:05:48.992352+00:00 app[web.1]: at com.fullstackryan.appone.server.ApponeServer$.$anonfun$stream$1$adapted(ApponeServer.scala:48)
2021-01-17T14:05:48.992496+00:00 app[web.1]: at fs2.Stream$.$anonfun$flatMap$1(Stream.scala:1188)
2021-01-17T14:05:48.992649+00:00 app[web.1]: at fs2.internal.FreeC$.go$2(Algebra.scala:609)
2021-01-17T14:05:48.992861+00:00 app[web.1]: at fs2.internal.FreeC$.$anonfun$flatMapOutput$1(Algebra.scala:616)
2021-01-17T14:05:48.993129+00:00 app[web.1]: at fs2.internal.FreeC$$anon$1.cont(Algebra.scala:53)
2021-01-17T14:05:48.996922+00:00 app[web.1]: at fs2.internal.FreeC$ViewL$$anon$9$$anon$10.cont(Algebra.scala:242)
2021-01-17T14:05:48.997120+00:00 app[web.1]: at fs2.internal.FreeC$ViewL$.mk(Algebra.scala:231)
2021-01-17T14:05:48.997247+00:00 app[web.1]: at fs2.internal.FreeC$ViewL$.apply(Algebra.scala:220)
2021-01-17T14:05:48.997395+00:00 app[web.1]: at fs2.internal.FreeC.viewL(Algebra.scala:106)
2021-01-17T14:05:48.997537+00:00 app[web.1]: at fs2.internal.FreeC$.go$1(Algebra.scala:414)
2021-01-17T14:05:48.997707+00:00 app[web.1]: at fs2.internal.FreeC$.$anonfun$compile$8(Algebra.scala:464)
2021-01-17T14:05:48.998481+00:00 app[web.1]: at fs2.internal.FreeC$.$anonfun$compile$1(Algebra.scala:430)
2021-01-17T14:05:48.998648+00:00 app[web.1]: at map @ fs2.internal.CompileScope.interruptibleEval(CompileScope.scala:393)
2021-01-17T14:05:48.998778+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.go$1(Algebra.scala:490)
2021-01-17T14:05:48.998907+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.$anonfun$compile$5(Algebra.scala:450)
2021-01-17T14:05:48.999061+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.go$1(Algebra.scala:447)
2021-01-17T14:05:48.999202+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:05:48.999348+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:05:49.000021+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.$anonfun$acquireResource$4(CompileScope.scala:185)
2021-01-17T14:05:49.000182+00:00 app[web.1]: at flatten @ fs2.internal.ScopedResource$$anon$1.acquired(ScopedResource.scala:139)
2021-01-17T14:05:49.000285+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.$anonfun$acquireResource$1(CompileScope.scala:183)
2021-01-17T14:05:49.000409+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.acquireResource(CompileScope.scala:180)
2021-01-17T14:05:49.000547+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.$anonfun$compile$10(Algebra.scala:498)
2021-01-17T14:05:49.000665+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:05:49.000782+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:05:49.000911+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:05:49.001062+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.$anonfun$acquireResource$4(CompileScope.scala:185)
2021-01-17T14:05:49.001165+00:00 app[web.1]: at flatten @ fs2.internal.ScopedResource$$anon$1.acquired(ScopedResource.scala:139)
2021-01-17T14:05:49.171354+00:00 heroku[web.1]: Process exited with status 1
2021-01-17T14:05:49.208726+00:00 heroku[web.1]: State changed from starting to crashed
2021-01-17T14:05:49.211138+00:00 heroku[web.1]: State changed from crashed to starting
2021-01-17T14:05:55.214070+00:00 heroku[web.1]: Starting process with command `target/universal/stage/bin/appone -Dhttp.port=${PORT}`
2021-01-17T14:05:58.596071+00:00 app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
2021-01-17T14:05:58.994577+00:00 app[web.1]: Picked up JAVA_TOOL_OPTIONS: -Xmx300m -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8
2021-01-17T14:05:59.000000+00:00 app[api]: Build succeeded
2021-01-17T14:06:04.235862+00:00 app[web.1]: [ioapp-compute-0] INFO o.h.c.PoolManager - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false
2021-01-17T14:06:04.319171+00:00 app[web.1]: pureconfig.error.ConfigReaderException: Cannot convert configuration to a scala.runtime.Nothing$. Failures are:
2021-01-17T14:06:04.319174+00:00 app[web.1]: at 'appone.db-config':
2021-01-17T14:06:04.319195+00:00 app[web.1]: - (application.conf @ jar:file:/app/target/universal/stage/lib/com.fullstackryan.appone-0.0.1-SNAPSHOT.jar!/application.conf: 10) Key not found: 'username'.
2021-01-17T14:06:04.319196+00:00 app[web.1]: - (application.conf @ jar:file:/app/target/universal/stage/lib/com.fullstackryan.appone-0.0.1-SNAPSHOT.jar!/application.conf: 10) Key not found: 'password'.
2021-01-17T14:06:04.319206+00:00 app[web.1]:
2021-01-17T14:06:04.319341+00:00 app[web.1]: at com.fullstackryan.appone.config.LoadConfig$$anon$1.$anonfun$load$1(LoadConfig.scala:25)
2021-01-17T14:06:04.319403+00:00 app[web.1]: at cats.syntax.EitherOps$.leftMap$extension(either.scala:172)
2021-01-17T14:06:04.319496+00:00 app[web.1]: at com.fullstackryan.appone.config.LoadConfig$$anon$1.load(LoadConfig.scala:25)
2021-01-17T14:06:04.319683+00:00 app[web.1]: at com.fullstackryan.appone.server.ApponeServer$.$anonfun$stream$1(ApponeServer.scala:50)
2021-01-17T14:06:04.319688+00:00 app[web.1]: at com.fullstackryan.appone.server.ApponeServer$.$anonfun$stream$1$adapted(ApponeServer.scala:48)
2021-01-17T14:06:04.319785+00:00 app[web.1]: at fs2.Stream$.$anonfun$flatMap$1(Stream.scala:1188)
2021-01-17T14:06:04.319847+00:00 app[web.1]: at fs2.internal.FreeC$.go$2(Algebra.scala:609)
2021-01-17T14:06:04.319951+00:00 app[web.1]: at fs2.internal.FreeC$.$anonfun$flatMapOutput$1(Algebra.scala:616)
2021-01-17T14:06:04.320042+00:00 app[web.1]: at fs2.internal.FreeC$$anon$1.cont(Algebra.scala:53)
2021-01-17T14:06:04.320188+00:00 app[web.1]: at fs2.internal.FreeC$ViewL$$anon$9$$anon$10.cont(Algebra.scala:242)
2021-01-17T14:06:04.320257+00:00 app[web.1]: at fs2.internal.FreeC$ViewL$.mk(Algebra.scala:231)
2021-01-17T14:06:04.320336+00:00 app[web.1]: at fs2.internal.FreeC$ViewL$.apply(Algebra.scala:220)
2021-01-17T14:06:04.320405+00:00 app[web.1]: at fs2.internal.FreeC.viewL(Algebra.scala:106)
2021-01-17T14:06:04.320481+00:00 app[web.1]: at fs2.internal.FreeC$.go$1(Algebra.scala:414)
2021-01-17T14:06:04.320561+00:00 app[web.1]: at fs2.internal.FreeC$.$anonfun$compile$8(Algebra.scala:464)
2021-01-17T14:06:04.320641+00:00 app[web.1]: at fs2.internal.FreeC$.$anonfun$compile$1(Algebra.scala:430)
2021-01-17T14:06:04.320719+00:00 app[web.1]: at map @ fs2.internal.CompileScope.interruptibleEval(CompileScope.scala:393)
2021-01-17T14:06:04.320785+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.go$1(Algebra.scala:490)
2021-01-17T14:06:04.320862+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.$anonfun$compile$5(Algebra.scala:450)
2021-01-17T14:06:04.320929+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.go$1(Algebra.scala:447)
2021-01-17T14:06:04.320995+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:06:04.321076+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:06:04.321138+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.$anonfun$acquireResource$4(CompileScope.scala:185)
2021-01-17T14:06:04.321230+00:00 app[web.1]: at flatten @ fs2.internal.ScopedResource$$anon$1.acquired(ScopedResource.scala:139)
2021-01-17T14:06:04.321289+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.$anonfun$acquireResource$1(CompileScope.scala:183)
2021-01-17T14:06:04.321385+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.acquireResource(CompileScope.scala:180)
2021-01-17T14:06:04.321473+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.$anonfun$compile$10(Algebra.scala:498)
2021-01-17T14:06:04.321531+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:06:04.321607+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:06:04.321668+00:00 app[web.1]: at flatMap @ fs2.internal.FreeC$.interruptGuard$1(Algebra.scala:429)
2021-01-17T14:06:04.321779+00:00 app[web.1]: at flatMap @ fs2.internal.CompileScope.$anonfun$acquireResource$4(CompileScope.scala:185)
2021-01-17T14:06:04.321811+00:00 app[web.1]: at flatten @ fs2.internal.ScopedResource$$anon$1.acquired(ScopedResource.scala:139)
2021-01-17T14:06:04.453054+00:00 heroku[web.1]: Process exited with status 1
2021-01-17T14:06:04.497636+00:00 heroku[web.1]: State changed from starting to crashed
2021-01-17T14:28:36.868097+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=appone2021.herokuapp.com request_id=a2e9c2cb-9e29-4e90-a528-2b93822a5b22 fwd="2.221.116.154" dyno= connect= service= status=503 bytes= protocol=https
2021-01-17T14:28:37.225963+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=appone2021.herokuapp.com request_id=d1e00110-d2b1-4512-9391-2ab54ac11947 fwd="2.221.116.154" dyno= connect= service= status=503 bytes= protocol=https
2021-01-17T14:42:04.309046+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=appone2021.herokuapp.com request_id=b6446286-f99c-4490-9ed4-ed8071f2d5c1 fwd="2.221.116.154" dyno= connect= service= status=503 bytes= protocol=https
2021-01-17T14:42:04.481028+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=appone2021.herokuapp.com request_id=c6bff236-696b-491d-9831-930f1d114fd8 fwd="2.221.116.154" dyno= connect= service= status=503 bytes= protocol=https
2021-01-17T14:45:18.000000+00:00 app[api]: Build started by user fullstackryan@gmail.com
2021-01-17T14:46:57.165052+00:00 app[api]: Release v25 created by user fullstackryan@gmail.com
2021-01-17T14:46:57.165052+00:00 app[api]: Deploy 5ab4cf89 by user fullstackryan@gmail.com
2021-01-17T14:46:58.334991+00:00 heroku[web.1]: State changed from crashed to starting
2021-01-17T14:47:02.441681+00:00 heroku[web.1]: Starting process with command `target/universal/stage/bin/appone -Dhttp.port=${PORT}`
2021-01-17T14:47:04.351208+00:00 app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
2021-01-17T14:47:04.458310+00:00 app[web.1]: Picked up JAVA_TOOL_OPTIONS: -Xmx300m -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8

Answer 1

你似乎拥有大部分的碎片，但我认为你没有正确地将它们组合在一起。

Heroku Postgres sets the DATABASE_URL environment variable for you:

作为配置过程的一部分，DATABASE_URL 配置变量会添加到您的应用配置中。这包含您的应用用于访问数据库的 URL。

您似乎已经在 application.conf 中使用了此环境变量：

db-config {
    driver = "org.postgresql.Driver"
    url = ${?DATABASE_URL}
    username = ${?DATABASE_USERNAME}
    password = ${?DATABASE_PASSWORD}
    connection-threads = 4
    pool-size = 10
}

问题是您也依赖于名为DATABASE_USERNAME 和DATABASE_PASSWORD 的环境变量，而Heroku 不 提供了这些环境变量。这就是导致您的应用程序失败的原因：

pureconfig.error.ConfigReaderException: Cannot convert configuration to a scala.runtime.Nothing$. Failures are:
at 'appone.db-config':
- (application.conf @ jar:file:/app/target/universal/stage/lib/com.fullstackryan.appone-0.0.1-SNAPSHOT.jar!/application.conf: 10) Key not found: 'username'.
- (application.conf @ jar:file:/app/target/universal/stage/lib/com.fullstackryan.appone-0.0.1-SNAPSHOT.jar!/application.conf: 10) Key not found: 'password'.

您可以尝试使用heroku config:set 设置它们，但这不是一个好主意，因为

您应用的 DATABASE_URL 配置变量的值可能随时更改。您不应在 Heroku 应用内部或外部依赖此值。

我建议只在您的application.conf 中设置url。然后，在您的应用程序代码中，您可以解析 URL 并连接到您的数据库，就像您已经尝试做的那样。由于它是 URL，因此您当前实例化 URI 的方法非常合适。

旁注：您的应用程序代码当前再次直接从环境中获取DATABASE_URL。我怀疑从您从application.conf 获得的任何配置对象中检索它会更惯用，您在其中设置了db-config.url。但我在 Scala 方面的经验不足，无法展示正确的方法。

Answer 2

下面一行

   config <- Stream.eval(LoadConfig[F, Config].load)

正在读取我的 application.conf 文件，该文件在开发模式下很好，因为我通过运行 source meta/dev.env 获取 application.conf 中提到的变量。 meta/dev.env 是我存储 application.conf 中提到的环境变量的位置。

但是在生产环境（特别是 Heroku 环境）中，我不能source meta/prd.env，因为我无法将变量静态存储在该位置/文件中，因为 Heroku 会经常更新其变量，这意味着这些变量会过时。

考虑到上述情况，我必须编写这样的代码，即每次运行应用程序时，我的应用程序都会获得 Heroku 注入的 DATABASE_URL 变量。我为此创建了一个函数：

  def prodConfig(): Config = {
    val dbUri = new URI(System.getenv("DATABASE_URL"))
    val username = dbUri.getUserInfo.split(":")(0)
    val password = dbUri.getUserInfo.split(":")(1)
    val dbUrl = "jdbc:postgresql://" + dbUri.getHost + dbUri.getPath

    Config(ServerConfig(5432, dbUri.getHost), DbConfig(dbUrl, username, password, 10))
  }

上面的代码通过使用 System.getenv("DATABASE_URL") 得到DATABASE_URL。然后该函数将其拆分为各个部分，例如用户名、密码等，然后我可以传递数据库配置等。

在开发中，我取消注释下面的第一行并注释掉下面的第二行。

//    config <- Stream.eval(LoadConfig[F, Config].load)
      conifg = prodConfig()

在 dev 中它从 application.conf 中读取，在 prod 中它将执行 System.getenv("DATABASE_URL") ，这依赖于 heroku 来确保它的提供。

这不是最好的解决方案，因为它依赖于在开发模式下的评论和取消评论，但现在它是我所拥有的最好的。很想听听有关如何改进这一点的建议。

希望这可以帮助其他陷入困境的人。

Answer 3

我建议改用JDBC_DATABASE_URL。它将为您的应用自动设置：https://devcenter.heroku.com/articles/connecting-to-relational-databases-on-heroku-with-java#using-the-database_url-in-plain-jdbc