Question : org.apache.hadoop.fs.unsupportedfilesystemexception: no filesystem for scheme "s3" spark
Answered by : thoughtful-tapir-ovwif8x0xxs1
os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages com.amazonaws:aws-java-sdk-pom:1.11.538,org.apache.hadoop:hadoop-aws:2.7.3 pyspark-shell'
Source : https://stackoverflow.com/questions/54358250/how-to-get-csv-on-s3-with-pyspark-no-filesystem-for-scheme-s3n | Last Update : Fri, 14 Aug 20