Hadoop s3 configuration file missing


I’m using the Hadoop library to upload files in S3. Because of some metric configuration file is missing I’m getting this exception

My current configurations are

Where to add this configuration file? What to add to that configuration file?


don’t worry about it, it’s just an irritating warning. It’s only relevant when you have the s3a or abfs connectors running in a long-lived app where the metrics are being collected and fed to some management tooling.

Set the log level to warn in the log4j.properties file in your spark conf dir

Leave a Reply