.....and vice versa?

yes if you are not using system calls of linux and windows. if you want your assembly code to run on both operating system, you have to switch the os to real time and use bios calls. but i dont think that is a good thing to do, and i am not even sure if this operating systems would allow you. you might have some problems.

Linux and Windows run on several different machine architectures to which they have
been ported, assembly language programs are machine specific and cannot be garuanteed
to run or properly assemble under another version of Linux or Windows.
If a compatible assembler and library were used on systems of the same architecture,
using for instance calls to a linked standard library, it might be possible depending on the
program. As for the compiled executable Linux and Windows use a different executable
file format, used by the operating system loader for linking, allocating memory and relocation.
The BIOS is effectively disabled in protected mode on x86 based PCs.
NASM is a x86 assembler for Linux and Windows.

If I use the same code, but different assemblesrs for Linux and Windows .will it work?

No it wouldn't. Applications expect the operating system to do various things of their behalf, such as allocating memory, writing to disk, displaying stuff on the screen, getting keyboard input, and so on. Every operating system has its own way of making those services available to applications, and, apart from the core services, you can't even count on the same services being available under different operating systems.

High level languages generally hide operating system calls away from you, but in assembly language programs they have to be made explicitly.

I am still a very beginner for assmebly laguage. I am learning x32 Windows assembly from a book. would it be very hard if I wanted to learn 64bit and Linux assembly after learning that?

The assembly language itself would be the same, but the operating system services you need to call to get things done would be completely different.

The only difference between 32 bit and 64 bit assembly language is the width of the registers - plus you have got eight more of them to play with on the 64 bit processors.