Install Hadoop on MacOS (Macbook M1) | HDFS



In this video I have shown how you can install hadoop on your MacOS( Macbook M1, M1 Max or silicon chip ).

Install Hadoop on Macbook M1 : https://arjuncodes.medium.com/install-hadoop-on-macos-efe7c860c3ed

Hadoop “Permission denied (publickey,password,keyboard-interactive)” warning :
https://stackoverflow.com/questions/48978480/hadoop-permission-denied-publickey-password-keyboard-interactive-warning/49960886

Install HomeBrew on MacBook M1 : https://youtu.be/c-1tOojFDFM

Install Java On Macbook M1 : https://youtu.be/FsX0_RXMwvY

Timelapses :
00:00-00:40 – Introduction
00:40-00:54 – Install Hadoop using Homebrew
00:54-03:10 – Install Code command in PATH
03:10-03:50 – Add JAVA_HOME path to hadoop environement
03:50-05:05 – Make Changes on core-site.xml file
05:05-05:40 – Make Changes to hdfs-site.xml file
05:40-06:15 – Make Changes to mapred-site.xml file
06:15-07:05 – Make Changes to yarn-site.xml file
07:05-08:22 – Permission denied (publickey,password,keyboard-interactive)
08:22-09:30 – start-all.sh to start hdfs and jps to visualize
09:30-11:00 – Create directory inside hdfs
11:00-11:36 – Outro

➡️ Subscribe : https://www.youtube.com/arjuncodes

Stay Connected :
➡️ LinkedIn : https://np.linkedin.com/in/arjungautam1
➡️ Github : https://github.com/arjungautam1

Join Community :
➡️Discord : https://discord.gg/cn2x3jhMW4

Follow me on :
➡️ Medium: https://arjuncodes.medium.com/
➡️ Dev.to : https://dev.to/arjuncodes

#Hadoop #MacbookM1 #ArjunCodes

source

21 thoughts on “Install Hadoop on MacOS (Macbook M1) | HDFS”
  1. 10:15 – Your username should actually be arjuncodes, then "hadoop fs -put demo.csv" would work by default, without a destination. The hdfs:///user directory should also be created an HDFS superuser account rather than a normal user.

  2. I keep getting this error
    Failed to retrieve data from /webhdfs/v1/?op=LISTSTATUS: Server Error

  3. Hi! Please can you help me to install Hbase on mac. I have already tried all stuff i see on internet but , it doesn’t work.

  4. Thanks mate, it;'s really helpful. Just followed each and every step in the video and am able to install hadoop successfully without any issue.
    Can you please help installing hive as well in M1 macOS.
    Am getting issues while installing through homebrew.
    Error: openjdk@8: no bottle available!

    Much appreciated in advance, brother.

  5. Hi! I followed your guide and for the most part it works, which is great! However, when i try to run a map reduce jar example wordcount, it comes up with an error saying the bin/bash : bin/java : directory is not found. Did i do something wrong here?

  6. I found your video very helpful, one question, once hadoop is started when you shut down the pc how can I log in again to Hadoop when I turn on my pc? do I have to do the whole process again?

  7. Thanks for the helpful video. It seems to work except for the following error 'ERROR: Cannot set priority of resourcemanager process…' . after start-all.sh. Any ideas what could be causing this type of error?

  8. The DataNode is not showing when I list them with 'jps'. I followed the exact steps, and I am using the Java 11 as in the video.

Leave a Reply

Your email address will not be published.

Captcha loading...