-
问题:PyCharm中通过pyspark无法调起spark
2019-10-16 20:39:09,343 | Dummy-1:22492 | django.db.backends:90 | utils:execute | DEBUG | (0.000) SELECT @@SQL_AUTO_IS_NULL; args=None
2019-10-16 20:39:09,344 | Dummy-1:22492 | django.db.backends:90 | utils:execute | DEBUG | (0.000) SELECT VERSION(); args=None
System check identified no issues (0 silenced).
2019-10-16 20:39:09,923 | Dummy-1:22492 | django.db.backends:90 | utils:execute | DEBUG | (0.003) SHOW FULL TABLES; args=None
2019-10-16 20:39:09,927 | Dummy-1:22492 | django.db.backends:90 | utils:execute | DEBUG | (0.002) SELECTdjango_migrations.app,django_migrations.nameFROMdjango_migrations; args=()
October 16, 2019 - 20:39:09
Django version 1.10.1, using settings ‘mysite.settings’
Starting development server at http://0.0.0.0:8001/
Quit the server with CTRL-BREAK.
2019-10-16 20:39:22,058 | Thread-4:15288 | django.db.backends:90 | utils:execute | DEBUG | (0.000) SELECT @@SQL_AUTO_IS_NULL; args=None
2019-10-16 20:39:22,058 | Thread-4:15288 | django.db.backends:90 | utils:execute | DEBUG | (0.000) SELECT VERSION(); args=None
2019-10-16 20:39:22,070 | Thread-4:15288 | django.db.backends:90 | utils:execute | DEBUG | (0.000) SELECTdjango_session.session_key,django_session.session_data,django_session.expire_dateFROMdjango_sessionWHERE (django_session.session_key= ‘k4micifggqpl2xjc4wapt12wwq0j12b3’ ANDdjango_session.expire_date> ‘2019-10-16 20:39:22’); args=(‘k4micifggqpl2xjc4wapt12wwq0j12b3’, ‘2019-10-16 20:39:22’)
2019-10-16 20:39:22,073 | Thread-4:15288 | django.db.backends:90 | utils:execute | DEBUG | (0.001) SELECTauth_user.id,auth_user.password,auth_user.last_login,auth_user.is_superuser,auth_user.username,auth_user.first_name,auth_user.last_name,auth_user.email,auth_user.is_staff,auth_user.is_active,auth_user.date_joinedFROMauth_userWHEREauth_user.id= 1; args=(1,)
Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties
19/10/16 20:39:22 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[main,5,main]
java.util.NoSuchElementException: key not found: _PYSPARK_DRIVER_CALLBACK_HOST
at scala.collection.MapLikeclass.apply(MapLike.scala:141)
at scala.collection.AbstractMap.apply(Map.scala:58)
at org.apache.spark.api.python.PythonGatewayServerKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲main$1.apply$mc…runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
19/10/16 20:39:22 INFO ShutdownHookManager: Shutdown hook called
2019-10-16 20:39:23,355 | Thread-4:15288 | django.server:131 | basehttp:log_message | ERROR | “GET /cif/cifmas_echart_0002/ HTTP/1.1” 500 102233 -
解决方法:在pycharm中增加变量,指向 spark目录即可。
相关文章: