It looks like the center piece above the armrest is part of the right seat. See part #5.
And part #4 is the panel behind that. Doesn't look like either is structural.
I think that if you did it right the arm rest would hide the opening when closed.
I have a sawzall. Where do you live? lol