Question

如何运行含spark的python脚本

Answer 1

2~spark$ bin/spark-submit first.py
-----------first.py-------------------------------
from pyspark import SparkConf, SparkContext
conf = SparkConf().setMaster("local").setAppName("My App")
sc = SparkContext(conf = conf)
lines = sc.textFile("first.py")
pythonLines = lines.filter(lambda line: "Python" in line)
print "hello python"
print pythonLines.first()
print pythonLines.first()
print "hello spark!"
---------------------------------------------------
hello python
pythonLines = lines.filter(lambda line: "Python" in line)
pythonLines = lines.filter(lambda line: "Python" in line)
hello spark!

到spark的安装目录下/bin 下面 spark-submit ***.py 即可

Answer 2

前段间使用google博客空间觉般所现几篇文章转 执行python脚本需要python文件做操作即： python文件第行加#! /usr/bin/python即python解释器所目录另外种写#! ...