I'd go with Watsonsimler's suggestion because it reduces the chance of the user accidentally clicking the delete button in a row they don't want to delete.
You can't also have a checkBox cell in every row and a single delete button on your form (not in the data grid view). This will let the user select multiple rows to delete at once. If you use the checkBox option you can add buttons to do other functions such as exporting the selected rows to an Excel document or something like that.
Sorry I went off on one then but it's another option you could think about which could reduce the amount of work you'll need to do if something like that became a requirement in the future.
i am trying to create a library management system which is supposed to be used to store/keep infp for the library and the library members( book title, author, name and picture for borrower and other stuffs). The app has 6 buttons, delete, save, add,previsous,next, and upload( for upload ...