如何使用 Elastic APM、Metricbeat 和 Filebeat 在 Kubernetes (AKS) 中安装 ELK？

Question

我能够安装 elasticseach 和 kibana，两者都已启动并正在运行。在 Kibana 仪表板中设置 APM 服务器，并显示索引。

我在跟踪日志时收到以下 APM-Agent 错误。 错误 co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - 尝试连接到 APM 服务器时出错。有关与当前连接对应的 SSL 配置的一些详细信息记录在 INFO 级别。

错误 co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - 无法处理 JSON_WRITER 类型的事件并出现此错误：连接超时

APM 代理 Yaml 文件

apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
namespace: blogdemodeployments
spec:  
selector:
matchLabels:
app: azuretest
template:
metadata:
labels:
app: azuretest
spec:
containers:
      - name: apm-agent-container
image: dockerid/application-service
volumeMounts:
        - name: shared-data
mountPath: /usr/share/app/
ports:
        - containerPort: 6000
name: http
protocol: TCP
env:
        - name: SERVER_URL
value: "http://40.83.185.238:8200"
      - name: filebeat-container
image: docker.elastic.co/beats/filebeat:7.10.0
volumeMounts:
        - name: shared-data
mountPath: /usr/share/filebeat/filebeat.yml

volumes:
      - name: shared-data
azureFile:
secretName: storage-secret
shareName: myfileshare
readOnly: false
 
---
kind: Service
apiVersion: apps/v1
metadata:
name: apmfb
namespace: blogdemodeployments
spec:
type: LoadBalancer
selector:
app: apmfb
ports:
    - name: http
protocol: TCP
port: 6000
targetPort: 6000

ElasticSearch Yaml

---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
  labels:
    component: elasticsearch
spec:
  version: 7.10.2
  http:
    service:
      spec:
        type: LoadBalancer
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false

Kibana Yaml

---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: quickstart
spec:
  version: 7.10.2 #Make sure Kibana and Elasticsearch are on the same version.
  http:
    service:
      spec:
        type: LoadBalancer #Adds a External IP
    tls:
      selfSignedCertificate:
        disabled: true
  count: 1
  elasticsearchRef:
    name: quickstart

APM 服务器 Yaml

---
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
  name: apm-new-quickstart
  namespace: default
spec:
  version: 7.10.2
  count: 1
  http:
    service:
      spec:
        type: LoadBalancer
    tls:
      selfSignedCertificate:
        disabled: true
  config:
    output:
      elasticsearch:
        enabled: true
        hosts: ["https://52.224.33.53:9200"]
        username: "elastic"
        password: "XXXXXXXXXXXXXXXXXXXX"
    logging:
      level: debug
      to_files: false
      to_stderr: true
    apm-server:
      hosts: "0.0.0.0:8200"

Answer 1

该错误表明代理无法连接到 apm-server。 SERVER_URL 应该是 apm-agent-container 环境中的 ELASTIC_APM_SERVER_URL。

Answer 2

感谢您的回复，我可以将 apm-server 与代理连接，但在 kibana 仪表板中，我收到“尚未从代理收到任何数据”。我的应用程序运行良好

2021-03-25 08:47:06,605 [main] INFO co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
2021-03-25 08:47:06,676 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.19.0 as demo-service1 on Java 10.0.2 Runtime version: 10.0.2+13-Debian-2 VM version: 10.0.2+13-Debian-2 (Oracle Corporation) Linux 5.4.0-1040-azure
2021-03-25 08:47:06,676 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - VM Arguments: [-javaagent:/app/elastic-apm-agent-1.19.0.jar, -Delastic.apm.service_name=demo-service1, -Delastic.apm.application_packages=com.javadeveloperzone.log4j, -Delastic.apm.server_urls=, -Delastic.apm.enable_log_correlation=true]
2021-03-25 08:47:07,836 [main] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state

 . ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.0.3.RELEASE)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.cglib.core.ReflectUtils$1 (jar:file:/app/app.jar!/BOOT-INF/lib/spring-core-5.0.7.RELEASE.jar!/) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of org.springframework.cglib.core.ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2021-03-25 08:47:12,189 [main] INFO co.elastic.apm.agent.servlet.ServletVersionInstrumentation - Servlet container info = Apache Tomcat/8.5.31
2021-03-25 08:52:15,083 INFO [Log4JExample] c979238deffe75982fe43844c268c83e - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 08:52:15,083 INFO [Log4JExample] c979238deffe75982fe43844c268c83e - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 08:52:20,706 INFO [Log4JExample] 3cb508519b2d44f9c5cab6da497d0745 - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 08:52:20,706 INFO [Log4JExample] 3cb508519b2d44f9c5cab6da497d0745 - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 10:46:03,993 INFO [Log4JExample] 2828f071583f306e89591a4a2edf2080 - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]